search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 490 Regressions on 9/26/2023 5:24:05 PM #22657

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
140.61 μs 213.47 μs 1.52 0.28 True
220.84 μs 369.70 μs 1.67 0.13 True
8.32 μs 9.53 μs 1.15 0.35 False
56.11 μs 86.07 μs 1.53 0.22 True
122.17 μs 201.80 μs 1.65 0.15 True
142.16 μs 214.16 μs 1.51 0.19 True
341.82 μs 534.36 μs 1.56 0.17 True
9.97 μs 10.64 μs 1.07 0.21 False
376.68 μs 545.78 μs 1.45 0.17 True
98.00 μs 178.37 μs 1.82 0.28 True
57.86 μs 87.85 μs 1.52 0.23 True
7.48 μs 8.55 μs 1.14 0.35 False
97.29 μs 128.34 μs 1.32 0.15 True
10.53 μs 11.23 μs 1.07 0.37 False
130.60 μs 184.62 μs 1.41 0.19 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateForEach<Int32>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 213.46833267857144 > 147.9891339645452. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -52.61741660862143 (T) = (0 -211251.80991294674) / Math.Sqrt((21069887.758342154 / (29)) + (24601595.592034776 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.46106163355524626 = (144587.8839477163 - 211251.80991294674) / 144587.8839477163 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 369.7015878801432 > 231.74341569927375. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -63.1800245482786 (T) = (0 -355925.9730084584) / Math.Sqrt((53034570.35807459 / (29)) + (63104792.43165854 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.5509429947491686 = (229490.0420024926 - 355925.9730084584) / 229490.0420024926 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.Queue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.52829331522848 > 8.744867015171614. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/25/2023 2:54:53 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -4.8403278765818705 (T) = (0 -9101.253247932473) / Math.Sqrt((145020.88417886494 / (20)) + (229451.90884511732 / (38))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (38) - 2, .025) and -0.06531323234212123 = (8543.264996270731 - 9101.253247932473) / 8543.264996270731 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ConcurrentStack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 86.06883136296764 > 58.566150920334906. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -59.491658540176616 (T) = (0 -88408.74494048547) / Math.Sqrt((5422559.1383256 / (28)) + (2609725.5485454975 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.5585770665800128 = (56724.01245738901 - 88408.74494048547) / 56724.01245738901 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 201.79986108534325 > 127.63337818677326. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -68.43477265309173 (T) = (0 -200604.34659397506) / Math.Sqrt((8879182.13798802 / (29)) + (26831320.806378268 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.622339511147019 = (123651.27349462426 - 200604.34659397506) / 123651.27349462426 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 214.16378217948713 > 143.7836754740794. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -67.4844372504021 (T) = (0 -209036.2409219502) / Math.Sqrt((18892407.78974314 / (28)) + (11189553.46702912 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.4980606648514222 = (139537.90111876573 - 209036.2409219502) / 139537.90111876573 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 534.3565277777777 > 381.4518022858137. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -54.271952770301866 (T) = (0 -535443.2732870284) / Math.Sqrt((82963049.52668552 / (28)) + (198895639.3925713 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.4655245409509744 = (365359.4725473385 - 535443.2732870284) / 365359.4725473385 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.Dictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.637797470190987 > 10.412617260108977. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -9.142744911335038 (T) = (0 -10804.678871491822) / Math.Sqrt((12437.14456682631 / (18)) + (297378.253226535 / (38))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (38) - 2, .025) and -0.08470561496496247 = (9960.932000744579 - 10804.678871491822) / 9960.932000744579 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 545.7807877155172 > 403.6579110748627. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -64.54759608025299 (T) = (0 -552381.3298479156) / Math.Sqrt((110659776.58959614 / (29)) + (87678359.28770289 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.4400802669652156 = (383576.7647951932 - 552381.3298479156) / 383576.7647951932 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ConcurrentQueue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 178.37181704119854 > 105.29129739314517. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -66.94135928285093 (T) = (0 -177264.2089172744) / Math.Sqrt((11156719.17779721 / (29)) + (24718150.191188272 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.724196421421978 = (102809.7534102763 - 177264.2089172744) / 102809.7534102763 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ConcurrentBag(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 87.84686349485018 > 59.28566862365498. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -41.41921336924034 (T) = (0 -89466.18174038023) / Math.Sqrt((9805711.088161528 / (29)) + (6190393.724771307 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.5354595755218291 = (58266.71256387519 - 89466.18174038023) / 58266.71256387519 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.Stack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.552265473086877 > 7.861329347645312. IsChangePoint: Marked as a change because one of 8/25/2023 8:22:05 AM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -6.301615409233784 (T) = (0 -8298.602022473995) / Math.Sqrt((145762.14843623823 / (20)) + (128283.02825223483 / (39))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (39) - 2, .025) and -0.08471332355862936 = (7650.502526555765 - 8298.602022473995) / 7650.502526555765 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 128.3416643369438 > 101.61378363081978. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -51.642099469768134 (T) = (0 -130718.21876209884) / Math.Sqrt((6172166.6470314795 / (29)) + (4875833.24887268 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3224769834272005 = (98843.47357285753 - 130718.21876209884) / 98843.47357285753 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.IEnumerable(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.229682281009785 > 11.075143438980751. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -4.173600763474148 (T) = (0 -11668.129305536533) / Math.Sqrt((590080.9368008779 / (17)) + (273844.8115966357 / (38))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (38) - 2, .025) and -0.07901954444319581 = (10813.640369746605 - 11668.129305536533) / 10813.640369746605 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ImmutableList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 184.62104083710403 > 140.3133272626424. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -40.90725146103903 (T) = (0 -189007.99183777042) / Math.Sqrt((24847274.912940644 / (29)) + (25573038.109890107 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3993455810183233 = (135068.84532427412 - 189007.99183777042) / 135068.84532427412 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
100.16 μs 124.49 μs 1.24 0.08 True
93.10 μs 107.22 μs 1.15 0.07 True
205.11 μs 235.85 μs 1.15 0.07 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 124.49064156746032 > 109.01014373761508. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -31.88923095347819 (T) = (0 -126825.5100155493) / Math.Sqrt((7667916.340669825 / (30)) + (7900845.214691612 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.2235613044431128 = (103652.76309001305 - 126825.5100155493) / 103652.76309001305 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 107.21609008072407 > 96.71404010923258. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.547159057247114 (T) = (0 -109682.00368078661) / Math.Sqrt((5886931.996209322 / (29)) + (3860763.8720351113 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.17042670600281593 = (93711.12528298951 - 109682.00368078661) / 93711.12528298951 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 235.85423863636365 > 216.22594562916672. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -15.576111911946398 (T) = (0 -236012.1521318002) / Math.Sqrt((41059743.86473691 / (27)) + (24722465.215061728 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.11317719153733377 = (212016.6977243396 - 236012.1521318002) / 212016.6977243396 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
578.62 μs 732.61 μs 1.27 0.13 True
93.54 μs 125.56 μs 1.34 0.17 True
68.71 μs 75.97 μs 1.11 0.25 False
136.30 μs 178.09 μs 1.31 0.12 True
144.62 μs 195.85 μs 1.35 0.12 True
10.31 μs 11.15 μs 1.08 0.28 False
1.30 ms 1.63 ms 1.26 0.06 True
818.45 μs 1.06 ms 1.29 0.05 True
1.26 ms 1.51 ms 1.20 0.10 True

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CtorFromCollection<Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 732.6134160839159 > 596.6115995582934. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -44.92592837208083 (T) = (0 -748158.6217955883) / Math.Sqrt((187735074.99955347 / (30)) + (283513553.4565783 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.3165707212970554 = (568263.1473518715 - 748158.6217955883) / 568263.1473518715 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.SortedDictionaryDeepCopy(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 125.55886380328342 > 98.68758474738024. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.358682206195084 (T) = (0 -129946.16006631457) / Math.Sqrt((10599568.03094617 / (29)) + (14872934.560539333 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.35543499673336104 = (95870.4477746913 - 129946.16006631457) / 95870.4477746913 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 75.971095068438 > 73.42085317493712. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.894893313112757 (T) = (0 -77525.91088246928) / Math.Sqrt((3225842.3542476376 / (18)) + (5376347.1563580455 / (39))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (39) - 2, .025) and -0.09456532553982983 = (70828.03472166834 - 77525.91088246928) / 70828.03472166834 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ImmutableList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 178.087154981737 > 141.50309773706897. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -40.69566015893023 (T) = (0 -179538.59291358112) / Math.Sqrt((21596325.039385285 / (28)) + (7037025.537675104 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.295746239011622 = (138559.99539734784 - 179538.59291358112) / 138559.99539734784 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 195.8521059981685 > 151.48204518633167. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -60.775982053172044 (T) = (0 -196288.31562890255) / Math.Sqrt((9095805.318971992 / (29)) + (10481445.138439855 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.34119885000549965 = (146352.88095281148 - 196288.31562890255) / 146352.88095281148 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ImmutableQueue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.149519966077829 > 10.824295120614915. IsChangePoint: Marked as a change because one of 9/24/2023 12:29:05 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -4.736953592888097 (T) = (0 -11185.809245139146) / Math.Sqrt((192630.6008691411 / (15)) + (316340.9992951021 / (43))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (43) - 2, .025) and -0.06404019100427465 = (10512.58151685194 - 11185.809245139146) / 10512.58151685194 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.630296444888889 > 1.356678873046875. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.79867113812968 (T) = (0 -1640803.2645498605) / Math.Sqrt((1284525126.4394386 / (29)) + (780414992.9765724 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.24129599029569762 = (1321846.9062797774 - 1640803.2645498605) / 1321846.9062797774 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0579024947222222 > 852.91707790625. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -60.68081007823377 (T) = (0 -1074095.9604783785) / Math.Sqrt((198731746.9062848 / (29)) + (322712764.027964 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3150272195274311 = (816786.1049023503 - 1074095.9604783785) / 816786.1049023503 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.509819228846154 > 1.313939503389423. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -31.997518464146747 (T) = (0 -1538921.5610151272) / Math.Sqrt((1138689825.8863826 / (29)) + (687398000.4923524 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.2008781006097466 = (1281496.8981728777 - 1538921.5610151272) / 1281496.8981728777 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.28 μs 5.29 μs 1.23 0.07 True
3.48 μs 4.41 μs 1.26 0.06 True
3.26 μs 3.98 μs 1.22 0.09 True
3.06 μs 3.63 μs 1.19 0.04 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.286679261742335 > 4.433115848002274. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -26.318945446906373 (T) = (0 -5273.451402203562) / Math.Sqrt((16133.650994281026 / (27)) + (18289.13393372781 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.2119853689242852 = (4351.085035691552 - 5273.451402203562) / 4351.085035691552 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.408034719841494 > 3.6599673669601596. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -29.20497854800972 (T) = (0 -4354.669030146075) / Math.Sqrt((9833.860749384576 / (29)) + (11619.052006030692 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2231051956557392 = (3560.3389190178536 - 4354.669030146075) / 3560.3389190178536 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.9769655130327726 > 3.396240604850789. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -35.62786140177071 (T) = (0 -4068.133364172115) / Math.Sqrt((7413.1213731019625 / (28)) + (5595.853498825573 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.23056710832360497 = (3305.901268329942 - 4068.133364172115) / 3305.901268329942 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.6308391202629546 > 3.0565064019013537. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.17639141069095 (T) = (0 -3720.4844284773444) / Math.Sqrt((5632.388674824651 / (28)) + (3848.737463587257 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.23824337744990706 = (3004.6471446828764 - 3720.4844284773444) / 3004.6471446828764 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
277.30 μs 332.32 μs 1.20 0.06 True
272.32 μs 323.36 μs 1.19 0.04 True
271.18 μs 322.18 μs 1.19 0.04 True
273.59 μs 321.76 μs 1.18 0.03 True
290.44 μs 337.15 μs 1.16 0.04 True
287.65 μs 340.69 μs 1.18 0.02 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 332.3226953900709 > 289.1091733204707. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -23.48170574477367 (T) = (0 -325513.0278599348) / Math.Sqrt((62839111.79655906 / (27)) + (37217694.89779391 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.15884982183725624 = (280893.1940325641 - 325513.0278599348) / 280893.1940325641 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 323.35862937127973 > 290.7645563466283. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.832511282904473 (T) = (0 -328909.22757355694) / Math.Sqrt((55551618.30633899 / (29)) + (25118552.96653116 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.16432694202343578 = (282488.7200513965 - 328909.22757355694) / 282488.7200513965 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 322.18331436011897 > 285.9666950408821. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -26.04258599512133 (T) = (0 -327521.1572821294) / Math.Sqrt((70497931.00008671 / (28)) + (19058795.845700473 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.16506970096190327 = (281117.2215805817 - 327521.1572821294) / 281117.2215805817 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 321.7630114795919 > 288.6721854028514. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -25.000763074336867 (T) = (0 -326570.6866291548) / Math.Sqrt((74413514.9289908 / (29)) + (21684177.156402085 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.16192401889149866 = (281060.277022856 - 326570.6866291548) / 281060.277022856 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 337.15256201475154 > 301.58156341934983. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -26.942191514105552 (T) = (0 -340961.1861752688) / Math.Sqrt((58077528.27134862 / (30)) + (32919280.268405426 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.16073290354246875 = (293746.4640958149 - 340961.1861752688) / 293746.4640958149 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 340.6852356884059 > 305.1022315914352. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -26.398447930663664 (T) = (0 -341065.5096078094) / Math.Sqrt((66760426.94745834 / (29)) + (22013737.306124795 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.15663183220111074 = (294878.1972901004 - 341065.5096078094) / 294878.1972901004 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.97 ms 3.72 ms 1.25 0.04 True
2.26 ms 2.85 ms 1.26 0.08 True
1.54 ms 1.78 ms 1.16 0.06 True
1.48 ms 1.67 ms 1.13 0.04 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.7193226130884045 > 3.1184869521532472. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -45.925113332419855 (T) = (0 -3719086.1988774375) / Math.Sqrt((4306995621.796086 / (29)) + (2947661568.678386 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2427138641770631 = (2992713.211049795 - 3719086.1988774375) / 2992713.211049795 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.85240156627451 > 2.376036338390986. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.579503268301053 (T) = (0 -2768902.8512561624) / Math.Sqrt((4184330288.5668063 / (29)) + (3523985278.0815873 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.1938731413135393 = (2319260.5273033637 - 2768902.8512561624) / 2319260.5273033637 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.784053062244898 > 1.5746830305915738. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -31.702999374486602 (T) = (0 -1800796.3823325846) / Math.Sqrt((1384092404.2190971 / (29)) + (827849282.8188138 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.18311994045830915 = (1522074.2384199898 - 1800796.3823325846) / 1522074.2384199898 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6735154149999998 > 1.5214421418892048. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -33.64306588723706 (T) = (0 -1703772.2597421422) / Math.Sqrt((871044412.0127879 / (29)) + (545139819.1679381 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.16007818924756104 = (1468670.1944178666 - 1703772.2597421422) / 1468670.1944178666 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
847.60 μs 1.38 ms 1.63 0.14 True
842.25 μs 1.40 ms 1.66 0.08 True
65.14 μs 76.54 μs 1.17 0.16 False
346.33 μs 711.77 μs 2.06 0.23 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Sort<Int32>.LinqOrderByExtension(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3780794799733331 > 858.1866289634615. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -61.22519918698042 (T) = (0 -1408595.462885439) / Math.Sqrt((396813955.847539 / (28)) + (2219380908.0999093 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.7063534587598215 = (825500.4000807704 - 1408595.462885439) / 825500.4000807704 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<Int32>.LinqQuery(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3974646933066666 > 874.478689007. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -116.6950573366344 (T) = (0 -1390270.9516982038) / Math.Sqrt((267671471.81524512 / (29)) + (417175000.3964276 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.688895650600441 = (823183.4519817318 - 1390270.9516982038) / 823183.4519817318 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<Int32>.Array_ComparerClass(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 76.53781427142857 > 68.929455021. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.983680117307697 (T) = (0 -73367.85157444574) / Math.Sqrt((2615241.1032228484 / (19)) + (7970787.700807175 / (38))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (38) - 2, .025) and -0.10652775501491421 = (66304.5741436977 - 73367.85157444574) / 66304.5741436977 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<Int32>.Array_Comparison(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 711.7725199999999 > 366.8699221234616. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -89.80587409144259 (T) = (0 -725771.4088191224) / Math.Sqrt((168412844.31160638 / (28)) + (312117430.183081 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -1.0277227913295037 = (357924.3730565659 - 725771.4088191224) / 357924.3730565659 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Tests.Perf_SortedSet

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
36.62 μs 47.06 μs 1.28 0.12 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_SortedSet*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Tests.Perf_SortedSet.EnumerateViewBetween #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.05945676480051 > 38.0878414047464. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.14916619940797 (T) = (0 -46446.37407522162) / Math.Sqrt((901515.4891029313 / (29)) + (1111417.1420187857 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2669830428588009 = (36659.03370768147 - 46446.37407522162) / 36659.03370768147 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
368.75 ns 397.95 ns 1.08 0.10 False
1.76 μs 2.04 μs 1.16 0.12 True
915.49 ns 963.05 ns 1.05 0.15 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.Logging.FormattingOverhead.NoArguments #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 397.9486265876264 > 391.51594001384797. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.391246562626975 (T) = (0 -403.4224409474071) / Math.Sqrt((73.25954180687535 / (28)) + (49.92761397740477 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.07427155385058674 = (375.53115830107555 - 403.4224409474071) / 375.53115830107555 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments_DefineMessage #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.0420495701880697 > 1.783054533100319. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -25.472507766318888 (T) = (0 -2011.687469567284) / Math.Sqrt((1742.9516354092545 / (27)) + (2119.913116165467 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.17448279582244478 = (1712.8283843090076 - 2011.687469567284) / 1712.8283843090076 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 963.0477572761882 > 960.417062471685. IsChangePoint: Marked as a change because one of 9/26/2023 10:37:55 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -9.5937092786579 (T) = (0 -990.6029201120313) / Math.Sqrt((541.5861345262279 / (27)) + (734.9240022906183 / (30))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (30) - 2, .025) and -0.06911384193839323 = (926.5644885075895 - 990.6029201120313) / 926.5644885075895 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.41 ms 2.78 ms 1.15 0.04 True
2.61 ms 2.98 ms 1.14 0.04 True
708.86 μs 767.37 μs 1.08 0.15 False
526.51 μs 743.89 μs 1.41 0.12 True
492.07 μs 527.64 μs 1.07 0.17 False
15.68 μs 18.11 μs 1.16 0.27 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CreateAddAndClear<Int32>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.7766044951310858 > 2.522416084849468. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -31.151538954941262 (T) = (0 -2811659.3742762394) / Math.Sqrt((2351364944.30809 / (29)) + (2077641216.8785212 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.15864286130616556 = (2426683.379472592 - 2811659.3742762394) / 2426683.379472592 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9769337332185883 > 2.633447224505495. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -35.368634083544954 (T) = (0 -2943351.6515030125) / Math.Sqrt((1728991248.6393192 / (28)) + (2445514140.328916 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.16991077887932518 = (2515877.0263852878 - 2943351.6515030125) / 2515877.0263852878 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<Int32>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 767.3671307397958 > 750.6413350142045. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.37902131629078 (T) = (0 -781061.7563984167) / Math.Sqrt((181595080.55476192 / (19)) + (438655670.4437482 / (38))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (38) - 2, .025) and -0.08540524171420079 = (719603.8183534762 - 781061.7563984167) / 719603.8183534762 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 743.8881086101397 > 558.2036750323277. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -42.79362224158974 (T) = (0 -729191.0008797584) / Math.Sqrt((267581522.85183918 / (28)) + (248573270.6489707 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.33310050254044854 = (546988.7675311513 - 729191.0008797584) / 546988.7675311513 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<Int32>.ImmutableList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 527.6355648809523 > 520.9062965106671. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -14.299754483123401 (T) = (0 -541313.8588161388) / Math.Sqrt((81843256.95818023 / (19)) + (172036635.4657272 / (39))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (39) - 2, .025) and -0.08460114646823168 = (499090.25135996763 - 541313.8588161388) / 499090.25135996763 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<Int32>.LinkedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.109670557170542 > 16.387201683136613. IsChangePoint: Marked as a change because one of 8/26/2023 1:27:07 AM, 9/24/2023 11:07:16 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -7.653205428763249 (T) = (0 -16760.832988470145) / Math.Sqrt((131518.40435485373 / (16)) + (377937.17009968555 / (40))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (40) - 2, .025) and -0.06461575234742513 = (15743.55155981239 - 16760.832988470145) / 15743.55155981239 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Perf_Frozen<NotKnownComparable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
226.59 μs 295.00 μs 1.30 0.03 True
1.56 μs 1.86 μs 1.19 0.06 True
17.86 μs 22.03 μs 1.23 0.14 True
131.80 μs 166.73 μs 1.27 0.07 True
26.88 μs 35.47 μs 1.32 0.05 True
16.87 μs 21.56 μs 1.28 0.10 True
1.09 μs 1.38 μs 1.27 0.15 True
139.04 μs 180.25 μs 1.30 0.18 True
18.04 μs 22.80 μs 1.26 0.17 True
1.46 μs 1.76 μs 1.20 0.03 True
155.65 μs 193.73 μs 1.24 0.09 True

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen&lt;NotKnownComparable&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenSet(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 294.9996380624093 > 237.5094206046196. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -65.38266604533474 (T) = (0 -302224.50009795465) / Math.Sqrt((15162938.623966007 / (29)) + (19712853.789422024 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.31103596211223417 = (230523.42485787743 - 302224.50009795465) / 230523.42485787743 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.TryGetValue_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8585805703963167 > 1.6193375709607383. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.42277602291634 (T) = (0 -1880.1300995108343) / Math.Sqrt((453.29926846417357 / (28)) + (1976.6869087725372 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2021431205746143 = (1563.9819147425208 - 1880.1300995108343) / 1563.9819147425208 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenDictionary(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.03281907801419 > 18.594241987441492. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.605026824234365 (T) = (0 -22467.77138686892) / Math.Sqrt((197683.39805433567 / (29)) + (168428.16702705086 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.24950181929372134 = (17981.38349215753 - 22467.77138686892) / 17981.38349215753 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.Contains_True(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 166.73408679078017 > 138.53272436687163. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -54.48726745070899 (T) = (0 -167985.1180281374) / Math.Sqrt((7774384.443506502 / (29)) + (3817635.361707372 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.2598801968356291 = (133334.1999104806 - 167985.1180281374) / 133334.1999104806 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenSet(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.46894457099469 > 28.1194722739078. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -55.61372974397266 (T) = (0 -36264.68257972771) / Math.Sqrt((294200.43034355616 / (29)) + (473020.34116513457 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3323316090350371 = (27218.961356019317 - 36264.68257972771) / 27218.961356019317 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.Contains_True(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.564356777188333 > 17.392268314661955. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -60.15705142278225 (T) = (0 -21002.114964854478) / Math.Sqrt((75492.51951878057 / (29)) + (81654.1145882838 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2671889004877888 = (16573.783874503613 - 21002.114964854478) / 16573.783874503613 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenSet(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3842912995332797 > 1.1641453827104993. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.61633796245476 (T) = (0 -1413.8800279928544) / Math.Sqrt((929.9939542057031 / (28)) + (834.4261712920352 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2649693328046454 = (1117.7188184143954 - 1413.8800279928544) / 1117.7188184143954 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.TryGetValue_True(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 180.25333528325126 > 148.25350918105698. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -43.272022749873095 (T) = (0 -183966.183325915) / Math.Sqrt((11981423.263396459 / (29)) + (13909797.354187908 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.285764175694008 = (143079.25730363178 - 183966.183325915) / 143079.25730363178 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.TryGetValue_True(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.79978709854015 > 18.641642348067357. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -30.06721997508288 (T) = (0 -22661.69349222488) / Math.Sqrt((321331.14693915297 / (29)) + (366527.1461228964 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.26315693025961795 = (17940.52104639698 - 22661.69349222488) / 17940.52104639698 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.Contains_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7558126569244525 > 1.546276567132077. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -26.479874654828652 (T) = (0 -1771.5415374753059) / Math.Sqrt((633.7380492861494 / (29)) + (2647.3987860339153 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.18905061753739844 = (1489.8789936666315 - 1771.5415374753059) / 1489.8789936666315 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenDictionary(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 193.7285337552742 > 163.02869716870362. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -35.3635533449618 (T) = (0 -194553.47691984658) / Math.Sqrt((9534144.026203055 / (29)) + (21650232.243056636 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2322694646606987 = (157882.2510005279 - 194553.47691984658) / 157882.2510005279 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
486.75 ns 575.30 ns 1.18 0.17 False
780.17 ns 872.23 ns 1.12 0.12 False
772.41 ns 837.39 ns 1.08 0.13 False
781.37 ns 925.02 ns 1.18 0.10 False
518.29 ns 567.15 ns 1.09 0.15 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 575.2986807737442 > 509.44259192627914. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.67314369981321 (T) = (0 -580.5450652848481) / Math.Sqrt((262.7317591409694 / (28)) + (189.95447619989324 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.15642657982226127 = (502.0163626592497 - 580.5450652848481) / 502.0163626592497 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 872.2345575479882 > 813.6711085983311. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -17.785368907533737 (T) = (0 -880.3406061768751) / Math.Sqrt((390.33723981217094 / (27)) + (307.3521242431295 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.11250268153854814 = (791.3154914461852 - 880.3406061768751) / 791.3154914461852 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 837.3948528340705 > 799.3613558315596. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -16.495363631182755 (T) = (0 -846.645698841652) / Math.Sqrt((313.23658687883847 / (27)) + (341.01557868391905 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.10395694564658928 = (766.9191286674412 - 846.645698841652) / 766.9191286674412 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 925.0188282207358 > 823.1891169303925. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -16.347114919481154 (T) = (0 -872.9414957840032) / Math.Sqrt((375.4062846577272 / (29)) + (303.11386894742475 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.09960346676280532 = (793.8693557905132 - 872.9414957840032) / 793.8693557905132 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 567.1540126602988 > 525.4998707132668. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -17.95798725399253 (T) = (0 -579.8346421728047) / Math.Sqrt((264.6600630598199 / (29)) + (189.6430480370753 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.13970788195124878 = (508.7572450408018 - 579.8346421728047) / 508.7572450408018 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
837.60 ns 1.23 μs 1.46 0.10 True
4.81 μs 6.05 μs 1.26 0.10 True
1.55 μs 2.11 μs 1.36 0.09 True
2.16 μs 2.93 μs 1.35 0.11 True
524.25 ns 578.08 ns 1.10 0.12 False
1.76 μs 2.47 μs 1.40 0.12 True
1.22 μs 1.78 μs 1.46 0.14 True

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_0Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2256960692127632 > 897.3568115007398. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -56.721192718002506 (T) = (0 -1249.002005510052) / Math.Sqrt((790.802010672529 / (29)) + (492.0658095432471 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.4327607305310106 = (871.745001726245 - 1249.002005510052) / 871.745001726245 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_5Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.051922005504935 > 5.109813268155044. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -31.19637606616057 (T) = (0 -6017.149162729604) / Math.Sqrt((19166.169687527225 / (28)) + (13269.253725828163 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.21242993885985362 = (4962.884014880083 - 6017.149162729604) / 4962.884014880083 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_2Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1115767147544693 > 1.6116098052594483. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -56.87613059164907 (T) = (0 -2099.91797384734) / Math.Sqrt((1480.1844914669375 / (29)) + (1249.8999406646096 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3564744082947358 = (1548.0704693037364 - 2099.91797384734) / 1548.0704693037364 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_4Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9276708901444404 > 2.1752721910551625. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -45.11912524362216 (T) = (0 -2908.807968150536) / Math.Sqrt((4261.69669774414 / (29)) + (4247.667345138145 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.36184531495521105 = (2135.931251668037 - 2908.807968150536) / 2135.931251668037 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory_3Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 578.0765779647863 > 540.5411212763377. IsChangePoint: Marked as a change because one of 8/14/2023 10:44:26 PM, 9/25/2023 2:54:53 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -10.92483702519874 (T) = (0 -555.0385179935038) / Math.Sqrt((115.86419919786357 / (20)) + (178.2409144892871 / (38))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (38) - 2, .025) and -0.06806905456908151 = (519.6653864458578 - 555.0385179935038) / 519.6653864458578 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_3Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.469618876946513 > 1.8697410515815223. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -38.15034805607203 (T) = (0 -2477.0867880356377) / Math.Sqrt((3171.7278958016277 / (30)) + (5300.44554313141 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.3542697258928971 = (1829.0941166852451 - 2477.0867880356377) / 1829.0941166852451 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_1Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7780484790299966 > 1.295729559812686. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -40.503380906449806 (T) = (0 -1720.4449612164271) / Math.Sqrt((2150.2596076935138 / (27)) + (1524.2503334269643 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.3711512514480067 = (1254.744842627353 - 1720.4449612164271) / 1254.744842627353 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
22.26 μs 26.27 μs 1.18 0.05 True
21.91 μs 26.66 μs 1.22 0.06 True
24.26 μs 28.13 μs 1.16 0.07 True
22.04 μs 26.63 μs 1.21 0.05 True
22.34 μs 26.48 μs 1.19 0.06 True
5.54 μs 6.83 μs 1.23 0.09 True
22.15 μs 25.13 μs 1.13 0.07 True
24.10 μs 27.68 μs 1.15 0.05 True
21.90 μs 24.98 μs 1.14 0.06 True
5.49 μs 6.36 μs 1.16 0.08 True
5.44 μs 6.68 μs 1.23 0.10 True
5.62 μs 6.54 μs 1.16 0.13 True
22.51 μs 25.87 μs 1.15 0.05 True
22.94 μs 25.23 μs 1.10 0.06 True
24.21 μs 28.28 μs 1.17 0.02 True
24.42 μs 27.55 μs 1.13 0.05 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.271336430419044 > 23.522063697512017. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.530285487468735 (T) = (0 -25873.10501025226) / Math.Sqrt((372977.9155592139 / (29)) + (256129.96702290187 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.14207263374995094 = (22654.51797518248 - 25873.10501025226) / 22654.51797518248 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.65601205204778 > 23.893250007940164. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -24.208752622856434 (T) = (0 -26169.974694185385) / Math.Sqrt((342205.963769415 / (29)) + (232368.90370947614 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.14970243915010636 = (22762.389469688344 - 26169.974694185385) / 22762.389469688344 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.129535983115865 > 25.343391180677052. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.413797418474843 (T) = (0 -27775.49587286425) / Math.Sqrt((268915.23072698485 / (29)) + (169844.21959618296 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.13925984847395365 = (24380.299112682427 - 27775.49587286425) / 24380.299112682427 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.634927398345496 > 23.340835325321674. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -20.766358507385593 (T) = (0 -25726.301377045573) / Math.Sqrt((445314.6720703365 / (29)) + (219482.3086484784 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.1392325947362576 = (22582.13247752224 - 25726.301377045573) / 22582.13247752224 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.484257095704553 > 23.374008503031387. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.059966931076914 (T) = (0 -25864.3650500519) / Math.Sqrt((454732.31592987356 / (29)) + (204723.2942950762 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.13997297898860614 = (22688.577296804866 - 25864.3650500519) / 22688.577296804866 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.826679961622805 > 5.826753752971497. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -29.27171951834281 (T) = (0 -6657.825166607485) / Math.Sqrt((17054.818920422247 / (29)) + (19302.232827072152 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.1843741840169177 = (5621.386599315123 - 6657.825166607485) / 5621.386599315123 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.12947883087894 > 23.269346246426814. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -22.594984408695645 (T) = (0 -25617.313395718844) / Math.Sqrt((310615.47444965655 / (28)) + (249546.99410464134 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.14128234910414053 = (22446.078672667965 - 25617.313395718844) / 22446.078672667965 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.6844189793894 > 25.231462702764386. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -20.733393510845836 (T) = (0 -27833.334727883386) / Math.Sqrt((373538.1822340708 / (28)) + (368587.36732592946 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.13666101174433037 = (24486.92656852028 - 27833.334727883386) / 24486.92656852028 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.9839257022098 > 23.03910155315719. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -23.674201247870545 (T) = (0 -25476.553687314357) / Math.Sqrt((282218.8873325688 / (30)) + (219611.58428479373 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.13777170338409275 = (22391.621809137134 - 25476.553687314357) / 22391.621809137134 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.360532781762295 > 5.820821157919904. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.657710438828417 (T) = (0 -6556.505339984474) / Math.Sqrt((14537.211035798087 / (27)) + (20325.979048861107 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.17440166462837195 = (5582.847451139484 - 6556.505339984474) / 5582.847451139484 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "ILEmit") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.682881203606386 > 5.760627485776667. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -33.784246188577015 (T) = (0 -6673.100376323561) / Math.Sqrt((11453.7654517303 / (27)) + (19384.913521010225 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.20098655754686168 = (5556.348931959783 - 6673.100376323561) / 5556.348931959783 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.5415574718632765 > 5.7952441599305775. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -25.206666881649948 (T) = (0 -6654.528362121066) / Math.Sqrt((18615.008377649396 / (28)) + (30552.292075861365 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.186267184845676 = (5609.637059113936 - 6654.528362121066) / 5609.637059113936 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.86501530100335 > 23.347826863504437. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -22.48586812142623 (T) = (0 -25347.88958601545) / Math.Sqrt((345225.5620319767 / (29)) + (204395.42967764332 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.13911285446284338 = (22252.30756259741 - 25347.88958601545) / 22252.30756259741 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.22564785870158 > 23.675334364707762. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -30.184025966890086 (T) = (0 -26094.718523829655) / Math.Sqrt((212210.8516934913 / (28)) + (187832.8134845337 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.15893254988466113 = (22516.166731555382 - 26094.718523829655) / 22516.166731555382 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.28228909102659 > 25.218851159655465. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -25.874156859364206 (T) = (0 -28027.915940723047) / Math.Sqrt((376960.39339868014 / (29)) + (212536.89588413603 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.1515677491237057 = (24338.920538588463 - 28027.915940723047) / 24338.920538588463 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.554924796479646 > 25.12878931967417. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -23.08542203771898 (T) = (0 -27944.32678638496) / Math.Sqrt((357550.9125517762 / (29)) + (343823.4113178132 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.14741458950085048 = (24354.16722262642 - 27944.32678638496) / 24354.16722262642 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Document.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.57 μs 1.65 μs 1.05 0.15 False
1.56 μs 1.79 μs 1.14 0.12 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6530806389948431 > 1.6263885947225551. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -10.584526664624132 (T) = (0 -1704.0533600215404) / Math.Sqrt((2225.8332338386253 / (29)) + (1932.698737146422 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.08035756977001692 = (1577.304966155135 - 1704.0533600215404) / 1577.304966155135 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7870356172839503 > 1.678820502544351. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.28663595231435 (T) = (0 -1751.4576278861298) / Math.Sqrt((1880.2954832251778 / (27)) + (1933.9858301964073 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.08136395977708288 = (1619.6744972406727 - 1751.4576278861298) / 1619.6744972406727 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<StructRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.57 μs 6.87 μs 1.23 0.11 True
5.84 μs 7.58 μs 1.30 0.08 True
4.97 μs 6.26 μs 1.26 0.11 True
6.20 μs 8.32 μs 1.34 0.12 True
5.24 μs 6.84 μs 1.30 0.12 True
6.05 μs 7.36 μs 1.22 0.08 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;StructRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.867031054527075 > 5.8224045883012865. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.893615949708206 (T) = (0 -6957.768337070584) / Math.Sqrt((22794.680859259195 / (29)) + (16182.263000874082 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.24130385145995117 = (5605.209658285722 - 6957.768337070584) / 5605.209658285722 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.584933881888835 > 6.1322811561382045. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -40.160341079081824 (T) = (0 -7401.319925498339) / Math.Sqrt((14140.40282227109 / (28)) + (26665.381625972226 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2575386052438746 = (5885.560804761934 - 7401.319925498339) / 5885.560804761934 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.260718180166962 > 5.328080624480043. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -46.90796585778696 (T) = (0 -6387.477992760026) / Math.Sqrt((10431.134339233593 / (29)) + (11172.685344228163 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.25068720145352597 = (5107.174667923854 - 6387.477992760026) / 5107.174667923854 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.315531613639148 > 6.765656618285893. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -28.763467751087294 (T) = (0 -8055.114024498666) / Math.Sqrt((46263.092794441596 / (29)) + (28160.19391356878 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.22084371672923397 = (6597.989500309791 - 8055.114024498666) / 6597.989500309791 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.839295924652628 > 5.525308684021818. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -33.67334266219903 (T) = (0 -6766.912349915102) / Math.Sqrt((23178.627978776232 / (29)) + (26984.502865440787 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.2641263557179798 = (5353.034781140791 - 6766.912349915102) / 5353.034781140791 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.36470171088677 > 6.364138243716407. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -35.60621083043491 (T) = (0 -7638.570018448249) / Math.Sqrt((30792.689284388784 / (29)) + (14998.31973680847 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.22733603963819168 = (6223.698947763349 - 7638.570018448249) / 6223.698947763349 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.82 μs 6.14 μs 1.27 0.10 True
6.10 μs 7.61 μs 1.25 0.07 True
4.33 μs 5.16 μs 1.19 0.12 False
3.86 μs 4.72 μs 1.22 0.07 True
5.32 μs 6.34 μs 1.19 0.13 False
5.19 μs 6.74 μs 1.30 0.11 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.139542453420151 > 5.021867428052559. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -40.11833754974872 (T) = (0 -6079.260032296903) / Math.Sqrt((18442.890436322137 / (28)) + (9725.551468370653 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2627259085098855 = (4814.39399582044 - 6079.260032296903) / 4814.39399582044 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.606739829402076 > 6.447443660337941. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -29.990967162922374 (T) = (0 -7614.942077285311) / Math.Sqrt((34768.95385791731 / (28)) + (16923.09254495491 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.20230467478667674 = (6333.62095064333 - 7614.942077285311) / 6333.62095064333 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.162725666006165 > 4.685306652656299. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -16.6455391414358 (T) = (0 -5136.7906993782835) / Math.Sqrt((24038.660863824363 / (27)) + (11411.397309051954 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.1313583835129093 = (4540.374450957229 - 5136.7906993782835) / 4540.374450957229 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.720110487940255 > 4.201186624515861. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.639433559291632 (T) = (0 -4657.225614449676) / Math.Sqrt((17579.834699093455 / (28)) + (7066.258051289599 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.14219317496519285 = (4077.4412914799636 - 4657.225614449676) / 4077.4412914799636 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.3439587113600115 > 5.6111137483830635. IsChangePoint: Marked as a change because one of 9/26/2023 10:37:55 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -14.661444222961421 (T) = (0 -6186.662026599954) / Math.Sqrt((32527.65750719414 / (27)) + (17525.606565631668 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.11209569218544782 = (5563.066263157771 - 6186.662026599954) / 5563.066263157771 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.736479922279791 > 5.458284106166121. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -33.58062539509955 (T) = (0 -6638.065842183226) / Math.Sqrt((28821.14004126326 / (29)) + (18603.524733375343 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2571879150633239 = (5280.090400685143 - 6638.065842183226) / 5280.090400685143 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
457.38 μs 638.48 μs 1.40 0.17 True
165.39 μs 243.05 μs 1.47 0.14 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.TryGetValueFalse<Int32, Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 638.4778571428571 > 480.67425567908657. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -50.3046830562996 (T) = (0 -652424.9157893556) / Math.Sqrt((130072409.14673537 / (28)) + (267381045.47732297 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.40273705611831767 = (465108.49124835915 - 652424.9157893556) / 465108.49124835915 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 243.04992603550298 > 174.03615451095325. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -57.55144197291835 (T) = (0 -243208.2994289134) / Math.Sqrt((15955656.83140883 / (28)) + (34553340.458073206 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.45783303949459747 = (166828.63732682928 - 243208.2994289134) / 166828.63732682928 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
25.36 μs 41.94 μs 1.65 0.34 True
14.82 μs 21.85 μs 1.47 0.23 True
3.10 μs 3.41 μs 1.10 0.44 False
15.05 μs 21.64 μs 1.44 0.27 True
30.00 μs 45.26 μs 1.51 0.33 True
3.31 μs 3.92 μs 1.18 0.32 False
94.05 μs 154.85 μs 1.65 0.14 True
19.99 μs 29.37 μs 1.47 0.24 True
13.96 μs 17.40 μs 1.25 0.39 False
8.10 μs 10.62 μs 1.31 0.26 True
8.30 μs 9.86 μs 1.19 0.26 False
14.31 μs 19.18 μs 1.34 0.23 False
24.79 μs 40.14 μs 1.62 0.20 True
2.50 μs 2.91 μs 1.16 0.33 False
14.37 μs 17.32 μs 1.21 0.32 True
5.14 μs 5.61 μs 1.09 0.18 False
3.84 μs 4.11 μs 1.07 0.23 False
2.25 μs 2.37 μs 1.05 0.34 False
14.89 μs 21.37 μs 1.44 0.24 True
3.10 μs 3.66 μs 1.18 0.38 False
9.05 μs 10.98 μs 1.21 0.39 True
3.10 μs 3.60 μs 1.16 0.43 False
80.22 μs 137.41 μs 1.71 0.08 True
29.66 μs 41.92 μs 1.41 0.17 True
18.74 μs 29.26 μs 1.56 0.21 True
14.38 μs 21.40 μs 1.49 0.30 True
7.71 μs 10.79 μs 1.40 0.21 True
2.93 μs 3.26 μs 1.11 0.24 False
87.29 μs 144.10 μs 1.65 0.13 True
24.71 μs 34.82 μs 1.41 0.21 True
19.06 μs 29.01 μs 1.52 0.28 True
23.11 μs 31.60 μs 1.37 0.12 True
38.27 μs 48.84 μs 1.28 0.13 True
2.58 μs 2.87 μs 1.11 0.27 False
755.06 ns 1.01 μs 1.34 0.14 True
7.79 μs 10.26 μs 1.32 0.31 True
18.93 μs 29.19 μs 1.54 0.20 True
3.74 μs 4.22 μs 1.13 0.39 False
25.69 μs 36.98 μs 1.44 0.22 True
33.23 μs 47.88 μs 1.44 0.10 True
28.69 μs 40.39 μs 1.41 0.17 True
67.55 ns 81.91 ns 1.21 0.52 False
26.03 μs 35.28 μs 1.36 0.17 True
15.61 μs 16.87 μs 1.08 0.25 True
40.56 μs 56.12 μs 1.38 0.16 True
25.69 μs 36.23 μs 1.41 0.21 True
13.90 μs 17.47 μs 1.26 0.33 True
25.90 μs 37.49 μs 1.45 0.21 True
19.41 μs 29.47 μs 1.52 0.13 True
3.12 μs 3.83 μs 1.23 0.44 False
23.60 μs 26.99 μs 1.14 0.25 False
20.06 μs 29.90 μs 1.49 0.16 True
120.79 μs 208.46 μs 1.73 0.09 True
3.21 μs 3.67 μs 1.14 0.41 False
25.59 μs 36.00 μs 1.41 0.20 True
3.64 μs 4.17 μs 1.15 0.36 False
20.26 μs 31.18 μs 1.54 0.12 True
24.88 μs 36.39 μs 1.46 0.15 True
27.15 μs 43.78 μs 1.61 0.25 True
3.14 μs 3.64 μs 1.16 0.39 False
1.04 μs 1.13 μs 1.09 0.30 False
42.86 μs 64.29 μs 1.50 0.12 True
45.12 μs 68.15 μs 1.51 0.09 True
26.09 μs 35.99 μs 1.38 0.18 True
24.35 μs 38.19 μs 1.57 0.21 True
21.02 μs 32.22 μs 1.53 0.21 True
9.76 μs 10.63 μs 1.09 0.25 False
12.32 μs 14.48 μs 1.18 0.39 False
7.84 μs 10.97 μs 1.40 0.32 True
3.23 μs 3.49 μs 1.08 0.47 False
80.00 μs 141.22 μs 1.77 0.12 True
36.40 μs 55.11 μs 1.51 0.11 True
3.70 μs 4.23 μs 1.14 0.38 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### System.Linq.Tests.Perf_Enumerable.Skip_One(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 41.93626391941392 > 26.76141134918043. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -52.41972579599852 (T) = (0 -43290.63277042247) / Math.Sqrt((1016644.917892006 / (28)) + (2073403.1458594638 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.6599627556337484 = (26079.2795642543 - 43290.63277042247) / 26079.2795642543 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.854195839169 > 15.160855454382157. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -43.66808660843904 (T) = (0 -21381.967846929772) / Math.Sqrt((457129.1911767293 / (29)) + (161373.34512632233 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.4250197587980511 = (15004.681664880656 - 21381.967846929772) / 15004.681664880656 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Aggregate(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.4059255866284266 > 3.311381760234239. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -9.923145276537792 (T) = (0 -3668.6896415065507) / Math.Sqrt((10926.014406878538 / (18)) + (44282.0450700786 / (39))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (39) - 2, .025) and -0.12727625359329722 = (3254.4725658970137 - 3668.6896415065507) / 3254.4725658970137 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Where(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.641021195131046 > 15.997914023433891. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.39572591680352 (T) = (0 -21673.07958923241) / Math.Sqrt((577571.9385313066 / (28)) + (247094.75708721267 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.41759934436900453 = (15288.579015870971 - 21673.07958923241) / 15288.579015870971 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 45.26423152353427 > 31.456064535005556. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -44.879067247525015 (T) = (0 -45692.81114867971) / Math.Sqrt((1171687.1997706702 / (29)) + (1968202.3758430362 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.485409057294855 = (30761.096362165004 - 45692.81114867971) / 30761.096362165004 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.923458610084606 > 3.318334667163543. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.838261658588229 (T) = (0 -3674.8123694706246) / Math.Sqrt((10222.11208221477 / (19)) + (35015.45141536991 / (38))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (38) - 2, .025) and -0.1540216918606747 = (3184.35294188065 - 3674.8123694706246) / 3184.35294188065 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.OrderBy(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 154.85094880363036 > 96.95450051785149. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -58.79752335557319 (T) = (0 -157940.71581556098) / Math.Sqrt((7666960.080749887 / (29)) + (25640196.827372916 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.6637964034982182 = (94927.91034016087 - 157940.71581556098) / 94927.91034016087 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.37494088408048 > 20.784630122781394. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -46.709620785964475 (T) = (0 -29947.57587879463) / Math.Sqrt((931371.8241197786 / (27)) + (240182.20476713977 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.47621239623369327 = (20286.766291355372 - 29947.57587879463) / 20286.766291355372 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.40276018284699 > 15.065787910625081. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -14.60130019456272 (T) = (0 -17412.59561180195) / Math.Sqrt((571402.1244396418 / (29)) + (452383.12502259476 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.18702175564907422 = (14669.146145750787 - 17412.59561180195) / 14669.146145750787 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.619922577982697 > 8.397521936262432. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.24100273593708 (T) = (0 -11054.353324009051) / Math.Sqrt((87652.57111484006 / (29)) + (119060.59456168418 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3541254101278137 = (8163.4634734205665 - 11054.353324009051) / 8163.4634734205665 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.CastToBaseClass(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.863574664659907 > 8.852346982142993. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 2:54:53 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.541689064353642 (T) = (0 -9429.389304641498) / Math.Sqrt((64976.969290411245 / (20)) + (72242.33311836034 / (38))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (38) - 2, .025) and -0.0962978033699032 = (8601.120312068997 - 9429.389304641498) / 8601.120312068997 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.1791667302799 > 15.045685608402854. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -16.94427195389196 (T) = (0 -17280.830264554144) / Math.Sqrt((186929.63122884787 / (29)) + (671097.7235298237 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.20988636228093005 = (14283.019301065247 - 17280.830264554144) / 14283.019301065247 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 40.140338452960265 > 26.235146065137293. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -45.0452989896798 (T) = (0 -36633.5218122836) / Math.Sqrt((602191.3726166302 / (29)) + (1092564.0880919464 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.4229859414667848 = (25744.120686478826 - 36633.5218122836) / 25744.120686478826 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.905125383486728 > 2.5865396255327933. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.802121986860422 (T) = (0 -2800.1685337645263) / Math.Sqrt((2569.7545542564703 / (19)) + (21776.095013850205 / (38))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (38) - 2, .025) and -0.12634485963579203 = (2486.0667759161893 - 2800.1685337645263) / 2486.0667759161893 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.319294877136752 > 15.03817811422461. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 10:37:55 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -14.857500992526807 (T) = (0 -17460.11528103267) / Math.Sqrt((677058.7633520113 / (28)) + (321541.871837197 / (30))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (30) - 2, .025) and -0.18901218026256356 = (14684.555440951868 - 17460.11528103267) / 14684.555440951868 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: IEnumerable, input2: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.609407887386346 > 5.320048941871774. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -9.76298188630126 (T) = (0 -5569.711074065482) / Math.Sqrt((25371.71792594982 / (19)) + (23140.70549738995 / (39))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (39) - 2, .025) and -0.08340114432531547 = (5140.949964137246 - 5569.711074065482) / 5140.949964137246 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.ToList(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.109816681315794 > 4.080490262734911. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -2.6105039339492726 (T) = (0 -4206.722106503623) / Math.Sqrt((124550.64094708471 / (19)) + (18859.689512202196 / (39))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (39) - 2, .025) and -0.054922836971667936 = (3987.7059810172664 - 4206.722106503623) / 3987.7059810172664 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Average(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3669336751648906 > 2.25506245012058. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -7.133227491187581 (T) = (0 -2504.507225188143) / Math.Sqrt((8230.316063079925 / (17)) + (49812.16374195835 / (39))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (39) - 2, .025) and -0.13576141024292465 = (2205.134989268971 - 2504.507225188143) / 2205.134989268971 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.3666610213205 > 16.639455945013292. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -30.67647279331246 (T) = (0 -22511.17788329182) / Math.Sqrt((943374.2999862819 / (28)) + (360571.8564248989 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.41376633726130546 = (15922.841908161161 - 22511.17788329182) / 15922.841908161161 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.656299456457696 > 3.3178391726541743. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 8:25:02 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -7.8099343891544635 (T) = (0 -3614.3560485474604) / Math.Sqrt((11367.102605984788 / (18)) + (62525.75890294908 / (40))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (40) - 2, .025) and -0.11262876987373269 = (3248.4833633752232 - 3614.3560485474604) / 3248.4833633752232 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.97518394764957 > 8.513961929310078. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.167350647242667 (T) = (0 -10981.117932058132) / Math.Sqrt((206782.27298371683 / (30)) + (82971.65499298075 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.32334041197657093 = (8298.02961707826 - 10981.117932058132) / 8298.02961707826 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.60219928546313 > 3.251209341462755. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -10.903887727621134 (T) = (0 -3684.776364086337) / Math.Sqrt((8054.818231812675 / (19)) + (57496.356219393645 / (39))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (39) - 2, .025) and -0.1480085195272092 = (3209.711688902674 - 3684.776364086337) / 3209.711688902674 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 137.40684464285715 > 84.25373908649615. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -106.95755382299446 (T) = (0 -139034.236784606) / Math.Sqrt((2984408.362996471 / (28)) + (5311488.676759483 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.7067461743862559 = (81461.57810173652 - 139034.236784606) / 81461.57810173652 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 41.91615064564131 > 30.280900737551292. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -68.26746934733916 (T) = (0 -42128.542338398074) / Math.Sqrt((470993.8461172954 / (28)) + (517086.0954944617 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.43194205230427535 = (29420.56368175304 - 42128.542338398074) / 29420.56368175304 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.255997203504048 > 19.83568221387843. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -50.220359877660194 (T) = (0 -29147.99004355532) / Math.Sqrt((615019.4610362294 / (29)) + (474439.90742013406 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.5013823369706225 = (19414.102141609023 - 29147.99004355532) / 19414.102141609023 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Select(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.40335737004104 > 15.102094448047096. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.10294251138176 (T) = (0 -21471.02016160497) / Math.Sqrt((479344.759190931 / (27)) + (394865.4808917761 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.44105163210480214 = (14899.549525678249 - 21471.02016160497) / 14899.549525678249 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.792190150187581 > 8.121037563151539. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -28.853304871331574 (T) = (0 -10847.464128595819) / Math.Sqrt((70790.34855365542 / (29)) + (214731.76509015853 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3585658352725901 = (7984.496479273913 - 10847.464128595819) / 7984.496479273913 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2585894890490334 > 3.0797184155338453. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.494761317078007 (T) = (0 -3322.944692097921) / Math.Sqrt((2325.3015994726134 / (19)) + (29721.87566256624 / (38))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (38) - 2, .025) and -0.12750822452276223 = (2947.1578298281734 - 3322.944692097921) / 2947.1578298281734 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.OrderByThenBy(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 144.0973463541667 > 91.44547882727282. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -86.21068751594794 (T) = (0 -144155.4885610988) / Math.Sqrt((6626230.743811943 / (29)) + (5150719.108448338 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.6235670542751632 = (88789.36547862916 - 144155.4885610988) / 88789.36547862916 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: Range) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 34.82205055176565 > 25.92901692449807. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -41.39436525668167 (T) = (0 -35483.41668358475) / Math.Sqrt((826048.1345992327 / (29)) + (853425.4322503703 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.3952360864263128 = (25431.836969233056 - 35483.41668358475) / 25431.836969233056 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Where(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.01386350548446 > 20.249845284911004. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -45.10519581384244 (T) = (0 -29300.32414245038) / Math.Sqrt((777664.3233056634 / (29)) + (500874.9863292919 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.4776077277551997 = (19829.568830804506 - 29300.32414245038) / 19829.568830804506 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.60077756085193 > 24.798919985101286. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -41.329150795100745 (T) = (0 -31520.70615038467) / Math.Sqrt((456821.20193217625 / (27)) + (460656.344897041 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.3114381336396069 = (24035.22159517042 - 31520.70615038467) / 24035.22159517042 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.GroupBy(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 48.836067903645834 > 39.62276239138115. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -49.521345856605215 (T) = (0 -49290.980865345366) / Math.Sqrt((515414.16107029427 / (27)) + (982091.098049708 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.3007185831853978 = (37895.19232102774 - 49290.980865345366) / 37895.19232102774 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.870835650983667 > 2.652615904667031. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.865543617521086 (T) = (0 -2827.2401172520686) / Math.Sqrt((3145.156775849398 / (19)) + (18271.068267695122 / (39))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (39) - 2, .025) and -0.12941064448058942 = (2503.2880034102236 - 2827.2401172520686) / 2503.2880034102236 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.010408245825724 > 806.6542494149313. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -28.19633448203009 (T) = (0 -968.6818372773112) / Math.Sqrt((759.5631729487735 / (29)) + (403.8962388868158 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.22604475882548114 = (790.0868465889313 - 968.6818372773112) / 790.0868465889313 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.263053696080707 > 8.17117168437717. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -24.89005347534806 (T) = (0 -10744.607147230452) / Math.Sqrt((214476.11650736653 / (28)) + (118386.98584926354 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.33515033820658774 = (8047.488615898429 - 10744.607147230452) / 8047.488615898429 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Select(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.19326441102757 > 20.22168137936821. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -54.71572646936759 (T) = (0 -29324.694888263813) / Math.Sqrt((463868.4497796466 / (29)) + (442105.7887113938 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.4920698363088117 = (19653.701304497466 - 29324.694888263813) / 19653.701304497466 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.217290875135208 > 4.03677584441563. IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -9.866995993039074 (T) = (0 -4346.9025811614) / Math.Sqrt((26206.347212896253 / (17)) + (66047.44177228198 / (39))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (39) - 2, .025) and -0.14824565129579517 = (3785.690436759696 - 4346.9025811614) / 3785.690436759696 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Range) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 36.980318602941175 > 27.010217024294136. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -51.39813387306304 (T) = (0 -36445.19482207394) / Math.Sqrt((623479.3732032933 / (29)) + (499282.37715425726 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3840694498024308 = (26331.911904620334 - 36445.19482207394) / 26331.911904620334 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Prepend(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.88125644871795 > 34.99124873363052. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -54.182735825187976 (T) = (0 -47702.55240416627) / Math.Sqrt((620956.4897375444 / (29)) + (1184901.0288704964 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3955577111776513 = (34181.71245953858 - 47702.55240416627) / 34181.71245953858 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Distinct(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 40.387283398268394 > 30.384178421710903. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -52.17115863738515 (T) = (0 -40553.95264420729) / Math.Sqrt((718125.1387610808 / (29)) + (576863.1737112643 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.37334562435464047 = (29529.312887470925 - 40553.95264420729) / 29529.312887470925 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IList) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 81.9131407497588 > 73.95947124910107. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -7.237860984210906 (T) = (0 -81.14459883746854) / Math.Sqrt((20.504490917457158 / (18)) + (25.6162573906434 / (39))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (39) - 2, .025) and -0.13576509647099744 = (71.4448780734658 - 81.14459883746854) / 71.4448780734658 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.281227979446015 > 26.54344505292338. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -48.62101928638841 (T) = (0 -36009.0036127664) / Math.Sqrt((898419.611200779 / (28)) + (303439.0119152676 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.3860465552356172 = (25979.64944016267 - 36009.0036127664) / 25979.64944016267 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.866302242152464 > 14.70841733915357. IsChangePoint: Marked as a change because one of 9/19/2023 5:34:53 AM, 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -18.428319546190142 (T) = (0 -17631.367035369094) / Math.Sqrt((288926.27789365256 / (27)) + (524409.421184358 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.2155503315338542 = (14504.843261505082 - 17631.367035369094) / 14504.843261505082 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Append(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 56.11858496702639 > 41.17658685658696. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -44.42709384471333 (T) = (0 -56974.984362170326) / Math.Sqrt((2261536.739125761 / (28)) + (1602703.6067711818 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.40370557113451794 = (40588.98499357055 - 56974.984362170326) / 40588.98499357055 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 36.22848794306183 > 27.070464725767195. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -46.88411713352915 (T) = (0 -36676.754800010596) / Math.Sqrt((894461.5401996198 / (29)) + (548734.3462715913 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3989271336457915 = (26217.77354795175 - 36676.754800010596) / 26217.77354795175 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.46842426119677 > 14.702530132219696. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -17.695376234360975 (T) = (0 -17375.405118808478) / Math.Sqrt((174106.55349151918 / (28)) + (741636.1030612531 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.22187530409745967 = (14220.276865029899 - 17375.405118808478) / 14220.276865029899 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.49294005554518 > 27.388115139615905. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.84926938765727 (T) = (0 -37104.131975103606) / Math.Sqrt((1417201.7694771197 / (29)) + (769963.8791799813 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3750235980764362 = (26984.36014262573 - 37104.131975103606) / 26984.36014262573 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SkipHalfTakeHalf(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.46847578908208 > 20.499464897736672. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -46.587758154634 (T) = (0 -29047.42048283899) / Math.Sqrt((630231.2117298627 / (29)) + (420580.74702829885 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.4394716454367399 = (20179.22379709391 - 29047.42048283899) / 20179.22379709391 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.833062980053462 > 3.2792713954519748. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.613139359090008 (T) = (0 -3673.396521021423) / Math.Sqrt((15108.040851532882 / (19)) + (18716.050746206016 / (38))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (38) - 2, .025) and -0.1533797702600774 = (3184.8976510079615 - 3673.396521021423) / 3184.8976510079615 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Except(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.991346800465383 > 24.989612506372733. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -16.495529769824746 (T) = (0 -27627.22016246533) / Math.Sqrt((515735.28143530875 / (27)) + (824833.3288012128 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.14967579500733777 = (24030.444306509038 - 27627.22016246533) / 24030.444306509038 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.89817961165048 > 20.46428838248534. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -60.212148297521594 (T) = (0 -29193.5620710511) / Math.Sqrt((461912.7374248015 / (29)) + (248097.32945620478 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.4765013514540171 = (19772.120115096473 - 29193.5620710511) / 19772.120115096473 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.OrderByDescending(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 208.45868940476188 > 127.5008270966046. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -94.3460664367443 (T) = (0 -204034.7855787366) / Math.Sqrt((9637790.603663014 / (29)) + (12024484.169390539 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.6656766178327258 = (122493.63615622694 - 204034.7855787366) / 122493.63615622694 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.666366665599112 > 3.3902856120452993. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -10.97897042627358 (T) = (0 -3584.6775455625802) / Math.Sqrt((4180.707488058005 / (17)) + (36395.611870062094 / (38))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (38) - 2, .025) and -0.11889407327530876 = (3203.768463147945 - 3584.6775455625802) / 3203.768463147945 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Select(input: IList) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.99556570307495 > 26.519728474412286. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.98339936735292 (T) = (0 -36140.27464322541) / Math.Sqrt((792600.8742213878 / (30)) + (1146448.3497461472 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.39688579927128675 = (25872.032389532993 - 36140.27464322541) / 25872.032389532993 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.167959398316849 > 3.90677108476539. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -9.942950134879426 (T) = (0 -4300.650317900989) / Math.Sqrt((26602.245539457992 / (19)) + (44141.28233569141 / (38))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (38) - 2, .025) and -0.13252426248338217 = (3797.4023695268015 - 4300.650317900989) / 3797.4023695268015 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.181508785338593 > 22.259758082654322. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -59.84163842430875 (T) = (0 -30677.319515982144) / Math.Sqrt((268011.0639619618 / (28)) + (424397.37198808166 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.4357369574118828 = (21366.949814597177 - 30677.319515982144) / 21366.949814597177 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 36.39379368686869 > 26.3260865148168. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -40.804647589476424 (T) = (0 -36014.904293200416) / Math.Sqrt((716071.8634650961 / (28)) + (1086622.5401501541 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.3975806287891113 = (25769.464423962694 - 36014.904293200416) / 25769.464423962694 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Take_All(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 43.777430629960314 > 26.650856514298294. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -61.47437592257337 (T) = (0 -43188.233958390825) / Math.Sqrt((926626.6088206454 / (28)) + (1264599.8147048294 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.6507387936405999 = (26162.97267912503 - 43188.233958390825) / 26162.97267912503 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.6431115102468556 > 3.3563813921899217. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.538470984308358 (T) = (0 -3654.002416674) / Math.Sqrt((10645.628032879333 / (18)) + (24515.110904191515 / (39))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (39) - 2, .025) and -0.1361772880599034 = (3216.049515400406 - 3654.002416674) / 3216.049515400406 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1300962305552895 > 1.0866717893182642. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -10.43584135318866 (T) = (0 -1177.6188255494963) / Math.Sqrt((668.3652690386574 / (18)) + (4825.800957683616 / (39))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (39) - 2, .025) and -0.12663158189554027 = (1045.256359286654 - 1177.6188255494963) / 1045.256359286654 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Intersect(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 64.2870009469697 > 44.24044426510045. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -57.95347142439232 (T) = (0 -63827.72627626944) / Math.Sqrt((1295842.6003554084 / (29)) + (2383903.6417946774 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.4780432010092074 = (43183.93821823874 - 63827.72627626944) / 43183.93821823874 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Zip(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 68.15162955555556 > 47.57773406637385. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -59.15878604454042 (T) = (0 -66537.96278437799) / Math.Sqrt((1552465.798301029 / (28)) + (1869513.5847762849 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.44483339906421576 = (46052.34266281015 - 66537.96278437799) / 46052.34266281015 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.990274503504665 > 27.32389252792488. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.65621792459663 (T) = (0 -36981.817895841115) / Math.Sqrt((1118675.8276041918 / (27)) + (763299.8555322012 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.38718939677904085 = (26659.530401335516 - 36981.817895841115) / 26659.530401335516 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 38.18674134632301 > 24.908682363868778. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -74.74366053085573 (T) = (0 -38164.18362452093) / Math.Sqrt((691333.9615313746 / (28)) + (281615.0613037988 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.5704762804322574 = (24301.025173087382 - 38164.18362452093) / 24301.025173087382 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IOrderedEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 32.22200282340862 > 21.67501569718595. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -60.72447546255655 (T) = (0 -31745.79581563198) / Math.Sqrt((436800.7594639366 / (27)) + (500867.2354009754 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.5380833944173339 = (20639.84042143444 - 31745.79581563198) / 20639.84042143444 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.630273431838768 > 10.048605217027056. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 2:54:53 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.843957852445612 (T) = (0 -10662.500458822047) / Math.Sqrt((62313.206158092355 / (20)) + (91385.00202131878 / (36))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (36) - 2, .025) and -0.09113822243604912 = (9771.906289761533 - 10662.500458822047) / 9771.906289761533 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.482157157226863 > 13.416060140696612. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -7.374980103427996 (T) = (0 -14281.757401185207) / Math.Sqrt((381194.98966146476 / (19)) + (639886.3648225921 / (39))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (39) - 2, .025) and -0.1094054029661526 = (12873.344012027437 - 14281.757401185207) / 12873.344012027437 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.967695528642723 > 8.528532879349175. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -29.43604417490902 (T) = (0 -11082.254519313767) / Math.Sqrt((86575.79954989911 / (29)) + (192784.4037436829 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.35262379944550104 = (8193.15357592914 - 11082.254519313767) / 8193.15357592914 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.4921655413895056 > 3.361637064656872. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.452212749454356 (T) = (0 -3678.483344957952) / Math.Sqrt((13000.489105069948 / (18)) + (23194.10327059346 / (39))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (39) - 2, .025) and -0.14005237624393002 = (3226.5915335199384 - 3678.483344957952) / 3226.5915335199384 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 141.22443170454545 > 84.56347167535115. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -108.93248458533138 (T) = (0 -140310.03146706722) / Math.Sqrt((5196929.738169889 / (29)) + (3078501.0931594013 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.708611845184693 = (82119.31332590074 - 140310.03146706722) / 82119.31332590074 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AppendPrepend(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.11314085695007 > 38.89214986923209. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -54.884398588764064 (T) = (0 -54807.074356346755) / Math.Sqrt((1684182.886218264 / (29)) + (970948.4537304366 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.4388112627749355 = (38091.9136333727 - 54807.074356346755) / 38091.9136333727 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.233941061205016 > 3.9800990043747064. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -8.468095369351367 (T) = (0 -4223.581708519553) / Math.Sqrt((30726.198451658438 / (19)) + (31648.916189927502 / (39))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (39) - 2, .025) and -0.10964083388894465 = (3806.260169533612 - 4223.581708519553) / 3806.260169533612 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.67 μs 2.91 μs 1.09 0.10 False
473.09 ns 540.97 ns 1.14 0.11 True
724.32 ns 893.58 ns 1.23 0.18 True
1.18 μs 1.29 μs 1.09 0.32 False
428.23 ns 525.95 ns 1.23 0.07 True
1.72 μs 1.93 μs 1.12 0.27 False
1.18 μs 1.31 μs 1.11 0.23 False
3.04 μs 3.21 μs 1.06 0.11 False
536.38 ns 624.20 ns 1.16 0.09 True
564.22 ns 660.61 ns 1.17 0.16 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'
### Payloads [Baseline]() [Compare]() ### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.912957859476903 > 2.7791968653257717. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -10.783212357040268 (T) = (0 -2908.468496017387) / Math.Sqrt((6960.396806377214 / (29)) + (4393.735543467867 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.07916822752042194 = (2695.1020441920377 - 2908.468496017387) / 2695.1020441920377 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 0) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 540.9669248137476 > 500.78420517965407. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -33.41266851905352 (T) = (0 -555.8869394821751) / Math.Sqrt((71.86593914076732 / (28)) + (89.44561365311769 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.16670442223731544 = (476.45910042595517 - 555.8869394821751) / 476.45910042595517 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 0) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 893.5835421644618 > 744.7195769912394. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -23.028580152576982 (T) = (0 -846.7995152061084) / Math.Sqrt((269.462940991418 / (28)) + (532.7821325561401 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.1680741338179462 = (724.9535716010375 - 846.7995152061084) / 724.9535716010375 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2877712929254892 > 1.2124366165073917. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -10.621806675291799 (T) = (0 -1291.1253860916115) / Math.Sqrt((1926.7215388394711 / (27)) + (1095.6215800885852 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.09402643848071561 = (1180.159217984356 - 1291.1253860916115) / 1180.159217984356 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 0) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 525.9509008891818 > 451.0086863250145. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -30.4689084314101 (T) = (0 -519.2279483388153) / Math.Sqrt((162.1355291802617 / (29)) + (72.78053673772227 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.20050220732312488 = (432.50894931429383 - 519.2279483388153) / 432.50894931429383 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.928882037094763 > 1.828794573371321. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.594594116142758 (T) = (0 -1964.2669709740549) / Math.Sqrt((1295.8703480749575 / (18)) + (5925.311588009549 / (39))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (39) - 2, .025) and -0.11553071276280319 = (1760.8362983652962 - 1964.2669709740549) / 1760.8362983652962 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 0) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3063222909558894 > 1.2536891976956952. IsChangePoint: Marked as a change because one of 9/25/2023 11:21:57 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.606969389526961 (T) = (0 -1335.0602181568051) / Math.Sqrt((1361.6590077464414 / (22)) + (2506.362783683747 / (36))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (36) - 2, .025) and -0.11074372667207617 = (1201.951616829571 - 1335.0602181568051) / 1201.951616829571 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://name5", uriCount: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2096486213801607 > 3.172315962452934. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 10:37:55 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -6.442068771907407 (T) = (0 -3265.0923764236118) / Math.Sqrt((16927.002181455067 / (28)) + (3916.791284498577 / (30))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (30) - 2, .025) and -0.056516883839037434 = (3090.4308547908217 - 3265.0923764236118) / 3090.4308547908217 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 624.2048015253944 > 558.2514311385715. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -25.860121181973014 (T) = (0 -636.3984862630879) / Math.Sqrt((224.8841961561295 / (29)) + (161.6112454142767 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.1741851798946435 = (541.9915845984278 - 636.3984862630879) / 541.9915845984278 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "name5", hostPortCount: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 660.6085268168189 > 593.6641072626043. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -23.526078214116843 (T) = (0 -676.5911167512129) / Math.Sqrt((184.44657752607696 / (29)) + (285.3743505246682 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.1647756217634133 = (580.8767835704589 - 676.5911167512129) / 580.8767835704589 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.29 μs 2.55 μs 1.11 0.05 True
1.21 μs 1.30 μs 1.08 0.06 False
630.18 ns 717.34 ns 1.14 0.09 True
636.21 ns 689.32 ns 1.08 0.12 True
1.05 μs 1.14 μs 1.09 0.08 False
2.17 μs 2.38 μs 1.10 0.16 False
1.43 μs 1.64 μs 1.14 0.13 False
3.22 μs 3.63 μs 1.13 0.24 False
841.75 ns 1.01 μs 1.19 0.15 False
2.11 μs 2.27 μs 1.08 0.08 False
5.51 μs 6.11 μs 1.11 0.22 False
865.53 ns 943.35 ns 1.09 0.17 False
900.50 ns 971.72 ns 1.08 0.14 False
3.66 μs 4.10 μs 1.12 0.10 False
887.62 ns 959.21 ns 1.08 0.21 False
859.52 ns 945.44 ns 1.10 0.21 False
439.23 ns 516.37 ns 1.18 0.10 True
894.60 ns 953.97 ns 1.07 0.11 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'
### Payloads [Baseline]() [Compare]() ### System.Reflection.Attributes.IsDefinedClassMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.5514216734155757 > 2.3831630850029377. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -23.509330717579793 (T) = (0 -2579.1430401104703) / Math.Sqrt((2203.464972274324 / (29)) + (1979.2170429629386 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.12292620648360035 = (2296.8054581137226 - 2579.1430401104703) / 2296.8054581137226 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodBaseMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2985690439438184 > 1.2636311250262218. IsChangePoint: Marked as a change because one of 9/26/2023 10:37:55 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -15.292007215357819 (T) = (0 -1321.2069694548877) / Math.Sqrt((472.05548551051146 / (27)) + (847.4601883823549 / (30))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (30) - 2, .025) and -0.08491840048104748 = (1217.7938625329527 - 1321.2069694548877) / 1217.7938625329527 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedClassHit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 717.3379724094059 > 644.8474440867568. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -20.750762158644857 (T) = (0 -719.8630297353594) / Math.Sqrt((232.71209250076305 / (28)) + (356.2781444406767 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.150512785519728 = (625.6888569996825 - 719.8630297353594) / 625.6888569996825 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedClassHitInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 689.3248888028661 > 661.0509578652878. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -15.614022807800756 (T) = (0 -703.7714706607272) / Math.Sqrt((194.04645576059286 / (30)) + (463.5770554248361 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.11748068401230771 = (629.7840139248224 - 703.7714706607272) / 629.7840139248224 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodBaseMiss #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1415378917951344 > 1.0984850941619078. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.101725787901884 (T) = (0 -1146.0429807099406) / Math.Sqrt((280.8669875369902 / (29)) + (1186.0708369875276 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.08988713839939103 = (1051.524456369877 - 1146.0429807099406) / 1051.524456369877 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesMethodBaseMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.380464440375526 > 2.3064581638475437. IsChangePoint: Marked as a change because one of 9/26/2023 5:24:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -9.2396672751975 (T) = (0 -2333.3203256014663) / Math.Sqrt((2503.4563839091174 / (30)) + (2519.517379211315 / (28))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (28) - 2, .025) and -0.05501810563268211 = (2211.640078159797 - 2333.3203256014663) / 2211.640078159797 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesClassMiss #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6363272854950175 > 1.5012095886551888. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -10.30101157517 (T) = (0 -1551.1994025488143) / Math.Sqrt((1474.1678862395406 / (28)) + (1614.735177322083 / (28))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (28) - 2, .025) and -0.07497820322632189 = (1443.0054468948433 - 1551.1994025488143) / 1443.0054468948433 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesMethodBaseHitInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.633218020641144 > 3.430548703002393. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 4:14:13 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -9.981809226374686 (T) = (0 -3563.92147040486) / Math.Sqrt((7741.044741456315 / (22)) + (13450.777186322684 / (35))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (35) - 2, .025) and -0.08224240514714208 = (3293.0898414762337 - 3563.92147040486) / 3293.0898414762337 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodOverrideMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0050426635469063 > 897.8692398612493. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.236961026954893 (T) = (0 -947.9814031507882) / Math.Sqrt((223.15726224291626 / (29)) + (1200.598179443738 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.09057840074954228 = (869.246449864816 - 947.9814031507882) / 869.246449864816 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesMethodBaseMiss #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.2730351750986983 > 2.201682422913485. IsChangePoint: Marked as a change because one of 9/26/2023 6:21:18 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -9.3766473661839 (T) = (0 -2258.8160612855777) / Math.Sqrt((2100.570823673574 / (27)) + (3382.2381921994415 / (31))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (31) - 2, .025) and -0.06016570971239015 = (2130.625467879325 - 2258.8160612855777) / 2130.625467879325 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesMethodOverrideMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.114873206543161 > 5.975561304461179. IsChangePoint: Marked as a change because one of 9/26/2023 10:37:55 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -8.52806811348212 (T) = (0 -6124.566939503465) / Math.Sqrt((30620.548580265655 / (27)) + (26305.55621007553 / (30))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (30) - 2, .025) and -0.06660046744875023 = (5742.137873006087 - 6124.566939503465) / 5742.137873006087 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodBaseHit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 943.3463899263004 > 920.503336002898. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.728199774402317 (T) = (0 -965.6510595059865) / Math.Sqrt((369.05358633201973 / (29)) + (1047.4961513283101 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.1014696627488252 = (876.6932873086217 - 965.6510595059865) / 876.6932873086217 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodOverrideHit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 971.7196846877403 > 944.9799721379071. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.550644168743933 (T) = (0 -994.7820447844314) / Math.Sqrt((787.8015435506172 / (28)) + (1043.440578763821 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.11237627405462836 = (894.2855650439539 - 994.7820447844314) / 894.2855650439539 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesClassMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.103213433395046 > 3.89780248794167. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -16.843716613883185 (T) = (0 -4167.9722492499) / Math.Sqrt((6045.867467618044 / (28)) + (9313.239021279773 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.10333198367310939 = (3777.62297379822 - 4167.9722492499) / 3777.62297379822 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodOverrideHitInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 959.2087596839169 > 919.2676121887589. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.209773001675519 (T) = (0 -986.9441089889317) / Math.Sqrt((228.13053414348198 / (29)) + (1577.9107374370412 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.12232127561800683 = (879.3775280126187 - 986.9441089889317) / 879.3775280126187 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodBaseHitInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 945.4375716224695 > 905.7821356097669. IsChangePoint: Marked as a change because one of 8/31/2023 1:23:15 PM, 9/21/2023 9:38:47 AM, 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.491069587397822 (T) = (0 -939.0121334686304) / Math.Sqrt((455.4457775610832 / (29)) + (748.2647258182432 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.08558873440446348 = (864.9796223095087 - 939.0121334686304) / 864.9796223095087 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedClassMiss #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 516.3651427600272 > 456.4551721188192. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -29.274970329954254 (T) = (0 -527.2246343708555) / Math.Sqrt((96.97511004973624 / (29)) + (167.85766463512167 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.20163181200579994 = (438.7572208918107 - 527.2246343708555) / 438.7572208918107 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodOverrideMiss #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 953.9663306646831 > 924.8059160156517. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.060846210243414 (T) = (0 -947.7632900748647) / Math.Sqrt((318.2992623452182 / (29)) + (544.4010927215112 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.08225690343061202 = (875.7285696867165 - 947.7632900748647) / 875.7285696867165 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.71 ms 2.18 ms 1.27 0.04 True
1.69 ms 2.13 ms 1.26 0.04 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.175663718078382 > 1.8014718090517243. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -38.63408680616595 (T) = (0 -2180871.4174251575) / Math.Sqrt((847404963.5502061 / (29)) + (3138852887.615276 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.26213887334131647 = (1727917.159901461 - 2180871.4174251575) / 1727917.159901461 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.127069627320955 > 1.7682233184392957. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -51.73265719044552 (T) = (0 -2175713.394056689) / Math.Sqrt((1012312357.2448943 / (29)) + (1450752808.405008 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2806230721456333 = (1698949.082973624 - 2175713.394056689) / 1698949.082973624 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.76 μs 1.95 μs 1.11 0.20 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9548898375195451 > 1.7774424388357313. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -18.20964213139223 (T) = (0 -1968.7595970051527) / Math.Sqrt((3251.5669749287435 / (27)) + (2290.880812767326 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.1502402580153976 = (1711.607277945577 - 1968.7595970051527) / 1711.607277945577 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
346.19 μs 398.94 μs 1.15 0.03 True
356.15 μs 401.53 μs 1.13 0.03 True
372.39 μs 417.71 μs 1.12 0.04 True
363.10 μs 395.55 μs 1.09 0.03 True
370.61 μs 416.80 μs 1.12 0.01 True
359.80 μs 402.84 μs 1.12 0.04 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 398.939217032967 > 372.42165759027785. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -18.900070726924337 (T) = (0 -399098.8636347635) / Math.Sqrt((94799595.34647606 / (28)) + (28674647.941502403 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.10993718058409523 = (359568.87526259915 - 399098.8636347635) / 359568.87526259915 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 401.53308161630025 > 377.0102718582354. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -20.586178323907095 (T) = (0 -399881.4256943534) / Math.Sqrt((41821904.95962268 / (29)) + (62675273.86792208 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.10830736823954167 = (360803.72390696395 - 399881.4256943534) / 360803.72390696395 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 417.712640625 > 383.13167161337213. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -20.37442145092523 (T) = (0 -416695.31580883614) / Math.Sqrt((103187235.15358679 / (29)) + (26291332.966711905 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.11521979006064405 = (373644.118874699 - 416695.31580883614) / 373644.118874699 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 395.54567261904765 > 373.6959938955966. IsChangePoint: Marked as a change because one of 9/25/2023 2:54:53 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -15.71309644520526 (T) = (0 -392166.36313971796) / Math.Sqrt((53397724.12622958 / (20)) + (113773879.65658632 / (37))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (37) - 2, .025) and -0.10623791686632374 = (354504.5393586042 - 392166.36313971796) / 354504.5393586042 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 416.79850337837837 > 385.0683357236296. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -22.293873220339616 (T) = (0 -412001.06062988413) / Math.Sqrt((51447065.911902726 / (28)) + (36450645.80549825 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.10519853369063727 = (372784.6609188588 - 412001.06062988413) / 372784.6609188588 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 402.8408883928572 > 374.11667044806575. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -22.166526992775257 (T) = (0 -401550.41190037323) / Math.Sqrt((48547700.30185708 / (29)) + (48113263.53225149 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.11435934953099958 = (360341.9418245773 - 401550.41190037323) / 360341.9418245773 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
58.49 μs 73.40 μs 1.25 0.10 True
60.38 μs 78.85 μs 1.31 0.07 True
61.95 μs 79.57 μs 1.28 0.03 True
50.69 μs 66.72 μs 1.32 0.13 True
67.22 μs 84.84 μs 1.26 0.07 True
65.31 μs 82.23 μs 1.26 0.04 True
55.96 μs 73.77 μs 1.32 0.07 True
51.83 μs 69.07 μs 1.33 0.15 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 73.39712920796309 > 59.41421900891252. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -42.342054897234135 (T) = (0 -72369.99040205216) / Math.Sqrt((2205483.054493859 / (27)) + (1194686.2226585709 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.258012012349395 = (57527.2649955845 - 72369.99040205216) / 57527.2649955845 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.85007685132032 > 63.649787875188785. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -38.601989465795484 (T) = (0 -77492.91989725367) / Math.Sqrt((2637454.538699426 / (28)) + (1657449.2580470305 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2403750271516338 = (62475.39510304917 - 77492.91989725367) / 62475.39510304917 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 79.56918797008547 > 64.9800727771606. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -43.33030612562561 (T) = (0 -78170.64587452683) / Math.Sqrt((2337834.0642670374 / (28)) + (921606.745525197 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.23182615864928793 = (63459.15397692307 - 78170.64587452683) / 63459.15397692307 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 66.72111233080224 > 53.28713987065707. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -35.1667268639972 (T) = (0 -67089.86125654013) / Math.Sqrt((2466676.793962049 / (28)) + (2842858.743559322 / (28))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (28) - 2, .025) and -0.29576832718439705 = (51776.123747615544 - 67089.86125654013) / 51776.123747615544 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 84.83684150050692 > 70.77093111276974. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -44.26748485991911 (T) = (0 -83833.00471208317) / Math.Sqrt((2256319.7269913424 / (27)) + (1088219.0201049636 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.22511519501792115 = (68428.67107762618 - 83833.00471208317) / 68428.67107762618 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.22868174603177 > 68.88788223678898. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -51.18900435172087 (T) = (0 -82339.84473150647) / Math.Sqrt((2043545.9333808504 / (29)) + (654260.0051995462 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.23398158035474148 = (66726.96419652848 - 82339.84473150647) / 66726.96419652848 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 73.76829434108528 > 58.55859195971207. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -42.09805267816979 (T) = (0 -70375.73882793171) / Math.Sqrt((1648470.6124608587 / (27)) + (1215832.063599177 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.23756529439862872 = (56866.28345709182 - 70375.73882793171) / 56866.28345709182 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 69.06629767699114 > 54.872020867582876. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -50.35912758936066 (T) = (0 -67836.6891244027) / Math.Sqrt((1398268.965116294 / (28)) + (1150579.439478671 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2857247551995253 = (52761.43968611343 - 67836.6891244027) / 52761.43968611343 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
385.38 ns 542.51 ns 1.41 0.20 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.TryWriteThenTryRead #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 542.506090098856 > 411.4834554347815. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -44.578215012346014 (T) = (0 -548.3459619439151) / Math.Sqrt((142.51259584000667 / (29)) + (191.69563081395344 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3811772957821256 = (397.0134490470325 - 548.3459619439151) / 397.0134490470325 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.83 ms 2.28 ms 1.25 0.05 True
1.90 ms 2.30 ms 1.21 0.10 True
22.83 ms 24.79 ms 1.09 0.01 True
1.80 ms 1.99 ms 1.11 0.05 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.2844259495412844 > 1.901802227429519. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -45.82184255404472 (T) = (0 -2263236.5904380856) / Math.Sqrt((1366022901.5111494 / (28)) + (1189664055.2971907 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.23742229656998895 = (1828992.8965330198 - 2263236.5904380856) / 1828992.8965330198 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.301856699376947 > 1.9017309713955126. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.24958143553564 (T) = (0 -2291474.7533942075) / Math.Sqrt((2169961771.6562634 / (29)) + (1671837908.3757985 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2455567182645183 = (1839719.315702465 - 2291474.7533942075) / 1839719.315702465 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.78700890178571 > 23.923650289444442. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -22.84605199556894 (T) = (0 -24861310.760588698) / Math.Sqrt((121230809637.83554 / (29)) + (48805692212.38719 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.07610789367111925 = (23102990.79377149 - 24861310.760588698) / 23102990.79377149 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9907301724759614 > 1.8575830098192405. IsChangePoint: Marked as a change because one of 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -18.1582256496714 (T) = (0 -1952655.7681149025) / Math.Sqrt((420464797.86257684 / (19)) + (2882603801.0624976 / (39))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (39) - 2, .025) and -0.1002720401070602 = (1774702.7070913322 - 1952655.7681149025) / 1774702.7070913322 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Interop.StructureToPtr

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
830.42 ns 1.09 μs 1.32 0.19 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Interop.StructureToPtr*'
### Payloads [Baseline]() [Compare]() ### Interop.StructureToPtr.MarshalPtrToStructure #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.094691184355535 > 908.2530015409634. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -29.71875490214322 (T) = (0 -1084.576772197534) / Math.Sqrt((672.5949431627149 / (29)) + (957.0257777093587 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.25850543367679146 = (861.7974489223177 - 1084.576772197534) / 861.7974489223177 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Perf_Frozen<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.61 μs 2.33 μs 1.45 0.21 True
65.27 μs 98.32 μs 1.51 0.41 True
57.27 μs 95.52 μs 1.67 0.31 True
7.42 μs 11.84 μs 1.60 0.27 True
1.54 μs 2.24 μs 1.45 0.19 True
6.86 μs 11.62 μs 1.69 0.33 True
1.18 μs 1.35 μs 1.15 0.15 True

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3333389437462597 > 1.6983519801882956. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -46.928499224966004 (T) = (0 -2360.296169112454) / Math.Sqrt((2677.858025786065 / (27)) + (3650.9125542356105 / (28))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (28) - 2, .025) and -0.4311275168866132 = (1649.2563669289423 - 2360.296169112454) / 1649.2563669289423 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 98.31520734447005 > 64.07289187896008. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -50.87627888141784 (T) = (0 -98428.22477086316) / Math.Sqrt((6362787.3498948775 / (28)) + (9870243.871388847 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.6377797122590335 = (60098.57371788937 - 98428.22477086316) / 60098.57371788937 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 95.52301213649157 > 57.99815639871567. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -60.34364940517525 (T) = (0 -92368.46784381368) / Math.Sqrt((4846866.27521506 / (28)) + (5386746.89275838 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.642981184218129 = (56220.03996824254 - 92368.46784381368) / 56220.03996824254 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.836118510773133 > 7.938588517306237. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -50.660101698025 (T) = (0 -12093.024127921231) / Math.Sqrt((145464.343098148 / (27)) + (76392.6496592388 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.6005213640899449 = (7555.678042946533 - 12093.024127921231) / 7555.678042946533 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.2408608012339712 > 1.633669482350896. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.52510376606962 (T) = (0 -2274.350484024695) / Math.Sqrt((3902.3295382113656 / (30)) + (7360.965519005076 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.42335151038155394 = (1597.883915137039 - 2274.350484024695) / 1597.883915137039 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.620801588038635 > 7.231448677032573. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -51.577234059248475 (T) = (0 -11476.056258636208) / Math.Sqrt((142637.99760061537 / (29)) + (86493.0828757314 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.6652575033461592 = (6891.460471174149 - 11476.056258636208) / 6891.460471174149 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.ToFrozenSet(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3499446632219292 > 1.211353628476121. IsChangePoint: Marked as a change because one of 9/26/2023 6:21:18 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -16.165771154169814 (T) = (0 -1357.1216398264723) / Math.Sqrt((2179.4824395983546 / (27)) + (1080.411053316968 / (31))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (31) - 2, .025) and -0.146865330009182 = (1183.3312982053499 - 1357.1216398264723) / 1183.3312982053499 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
568.13 ns 737.58 ns 1.30 0.13 True
24.31 μs 35.19 μs 1.45 0.08 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Scoped #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 737.5766626639258 > 588.3003655237919. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.576291619060964 (T) = (0 -716.8866572006481) / Math.Sqrt((320.40782873000677 / (28)) + (460.352093017074 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.25164317769023614 = (572.7564133122829 - 716.8866572006481) / 572.7564133122829 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Transient #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.188715695488725 > 25.945710205578134. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -70.18687901376782 (T) = (0 -35248.2792948483) / Math.Sqrt((356799.77550374664 / (28)) + (229722.4917609477 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.40102881329111467 = (25158.85395108158 - 35248.2792948483) / 25158.85395108158 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.43 ms 12.75 ms 1.12 0.01 True
3.03 ms 4.34 ms 1.43 0.08 True
2.38 ms 3.12 ms 1.31 0.12 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.751969906015038 > 11.999470080721155. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -42.00326249532542 (T) = (0 -12808544.559998028) / Math.Sqrt((17357206465.909664 / (28)) + (9841083319.36566 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.11304790927211836 = (11507630.94139786 - 12808544.559998028) / 11507630.94139786 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.33775122906404 > 3.102454468871849. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -51.95958537278505 (T) = (0 -4246411.911371492) / Math.Sqrt((9272876389.344933 / (28)) + (6108392212.159521 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.3982441740774442 = (3036960.2034446215 - 4246411.911371492) / 3036960.2034446215 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.1159796997863256 > 2.588707721270396. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -32.503435080543056 (T) = (0 -3085515.302870269) / Math.Sqrt((3927835767.8534293 / (28)) + (5510518398.257784 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.23678120358360544 = (2494794.789838258 - 3085515.302870269) / 2494794.789838258 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
504.93 μs 652.03 μs 1.29 0.17 True
166.46 μs 242.01 μs 1.45 0.08 True
126.71 μs 138.78 μs 1.10 0.21 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsKeyFalse<Int32, Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 652.026539230769 > 503.2503060260052. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -45.29483487900404 (T) = (0 -674861.3056608132) / Math.Sqrt((157055815.22474122 / (29)) + (348063218.5927869 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.389105353377969 = (485824.4221863471 - 674861.3056608132) / 485824.4221863471 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsKeyFalse<Int32, Int32>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 242.011363490676 > 173.5254822842595. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -77.07786141036573 (T) = (0 -242001.83438347784) / Math.Sqrt((17406578.17563116 / (28)) + (9419531.932663554 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.4489814325588925 = (167015.13832106465 - 242001.83438347784) / 167015.13832106465 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 138.77536557017544 > 135.62242552015093. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -9.003406719633283 (T) = (0 -142497.62433711538) / Math.Sqrt((7684403.721263875 / (17)) + (72900546.75805332 / (39))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (39) - 2, .025) and -0.1065172991232812 = (128780.29512057289 - 142497.62433711538) / 128780.29512057289 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.25 μs 3.98 μs 1.22 0.13 True
2.76 μs 3.27 μs 1.18 0.15 True
3.07 μs 3.59 μs 1.17 0.14 True
2.28 μs 2.87 μs 1.26 0.16 True
2.95 μs 3.70 μs 1.26 0.12 True
2.86 μs 3.43 μs 1.20 0.18 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.977880734157001 > 3.4448190876630234. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -25.372409712019536 (T) = (0 -4043.9275850716763) / Math.Sqrt((12080.501869228276 / (29)) + (11408.99612907323 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.2196714104195097 = (3315.587748081063 - 4043.9275850716763) / 3315.587748081063 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.265154481231548 > 2.7273192982409062. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -31.272826020856385 (T) = (0 -3218.3120123307694) / Math.Sqrt((4870.943750645414 / (28)) + (6524.713537789219 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2408308409931125 = (2593.675065132132 - 3218.3120123307694) / 2593.675065132132 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.5895220853014607 > 3.1014859424732593. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -24.707562500839714 (T) = (0 -3656.1020711240963) / Math.Sqrt((10041.747384700484 / (29)) + (8263.00070777733 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.2064854548223073 = (3030.373931580113 - 3656.1020711240963) / 3030.373931580113 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.8720357628004183 > 2.395467675970837. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.956897317759875 (T) = (0 -2980.5688001788517) / Math.Sqrt((4719.857853360316 / (29)) + (3631.356651826413 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.27568312662048644 = (2336.4491839559823 - 2980.5688001788517) / 2336.4491839559823 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.699504553214033 > 3.125292342362276. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -31.033945743923677 (T) = (0 -3707.3105988447687) / Math.Sqrt((9211.839380603304 / (29)) + (4448.2240168103945 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.22201475099426288 = (3033.7691061653754 - 3707.3105988447687) / 3033.7691061653754 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.4301296778972783 > 2.867014018700232. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -31.947272653177755 (T) = (0 -3434.598353089149) / Math.Sqrt((5355.400943150879 / (28)) + (6686.425519995467 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2361568256522525 = (2778.4487225372063 - 3434.598353089149) / 2778.4487225372063 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.65 μs 3.43 μs 1.30 0.12 True
2.04 μs 2.64 μs 1.29 0.15 True
2.41 μs 3.16 μs 1.31 0.10 True
1.71 μs 2.21 μs 1.30 0.18 True
2.28 μs 2.91 μs 1.28 0.10 True
1.88 μs 2.45 μs 1.30 0.12 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.433623542539123 > 2.841260819870601. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -38.603996202495345 (T) = (0 -3400.196820910645) / Math.Sqrt((4844.691741143221 / (29)) + (2892.655990877526 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.22767070907541656 = (2769.6326024357145 - 3400.196820910645) / 2769.6326024357145 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.64365213706234 > 2.133802928923254. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -29.49812236896114 (T) = (0 -2631.777706175355) / Math.Sqrt((6925.152102426392 / (28)) + (2858.1031238044025 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.26334969391941326 = (2083.174372734863 - 2631.777706175355) / 2083.174372734863 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.159803809096185 > 2.5966959050053826. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.737261400821282 (T) = (0 -3060.5979709107746) / Math.Sqrt((9275.620524317284 / (29)) + (4390.313730430879 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.1822802627966254 = (2588.724575060639 - 3060.5979709107746) / 2588.724575060639 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.2145765596297866 > 1.7661388715819113. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -28.593251684184136 (T) = (0 -2185.9460101201776) / Math.Sqrt((5065.775111269982 / (29)) + (2141.683195713371 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.2615182814762473 = (1732.7897995755966 - 2185.9460101201776) / 1732.7897995755966 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9142828058477335 > 2.341642463251784. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.96693475328435 (T) = (0 -2940.870485065946) / Math.Sqrt((6113.023570669941 / (29)) + (3074.762361595914 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.288228464653965 = (2282.8796023040077 - 2940.870485065946) / 2282.8796023040077 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4490749594630516 > 1.9705788639757573. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -35.25116563904737 (T) = (0 -2504.309287456998) / Math.Sqrt((4483.075571806202 / (29)) + (3674.3339312655 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3090408138083903 = (1913.087247578794 - 2504.309287456998) / 1913.087247578794 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.05 ms 1.28 ms 1.22 0.07 True
1.09 ms 1.33 ms 1.21 0.11 True
2.34 ms 2.74 ms 1.17 0.09 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2799697864583333 > 1.0916156714453125. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -25.53186066139117 (T) = (0 -1251970.0345355442) / Math.Sqrt((1025006399.8964585 / (28)) + (647492291.7407582 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.18561735897893009 = (1055964.6626747756 - 1251970.0345355442) / 1055964.6626747756 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.327241072172619 > 1.1623100857752402. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.86168047785016 (T) = (0 -1361239.6773035815) / Math.Sqrt((777748860.4581778 / (29)) + (617703748.6553628 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.21603150582177313 = (1119411.520825424 - 1361239.6773035815) / 1119411.520825424 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.7439687589285713 > 2.499151692276786. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.42137265682968 (T) = (0 -2751609.4762959806) / Math.Sqrt((4082420666.708192 / (29)) + (3667684992.6786895 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.14582562377030414 = (2401420.791448086 - 2751609.4762959806) / 2401420.791448086 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
293.47 μs 346.87 μs 1.18 0.06 True
309.42 μs 420.92 μs 1.36 0.11 True
811.58 μs 879.40 μs 1.08 0.04 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 346.8710427150974 > 312.40893863020455. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -25.121746202219498 (T) = (0 -351576.82418457366) / Math.Sqrt((40037555.06477416 / (29)) + (68538663.76618746 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.16044342091531924 = (302967.6568869351 - 351576.82418457366) / 302967.6568869351 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 420.9151772442084 > 335.6807852163462. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -40.27651344673485 (T) = (0 -410956.9398868626) / Math.Sqrt((51471333.31250283 / (28)) + (86871096.12416275 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2746596705593342 = (322405.22657042276 - 410956.9398868626) / 322405.22657042276 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 879.4020408163266 > 821.6180858203126. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.08988054395601 (T) = (0 -890726.0529896187) / Math.Sqrt((126828284.30746865 / (29)) + (101862672.87683071 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.12953718339684578 = (788576.1231081807 - 890726.0529896187) / 788576.1231081807 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.92 μs 3.25 μs 1.11 0.03 True
2.17 μs 2.39 μs 1.10 0.03 False
1.42 μs 1.52 μs 1.07 0.02 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2503508536543007 > 3.0716528024015126. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.32188752984274 (T) = (0 -3217.724301169056) / Math.Sqrt((2983.198037109534 / (27)) + (2366.2631009562565 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.09077849764583702 = (2949.933747423222 - 3217.724301169056) / 2949.933747423222 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.390706482128078 > 2.2994886616808983. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -20.044347751002924 (T) = (0 -2389.404848457676) / Math.Sqrt((1074.8805826006344 / (29)) + (960.967745123712 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.07560091690546616 = (2221.460404972562 - 2389.404848457676) / 2221.460404972562 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5196126024604495 > 1.4829565974965884. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -15.812662378792904 (T) = (0 -1539.828182863033) / Math.Sqrt((654.4174698853201 / (28)) + (574.1120593681662 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.07235301023132908 = (1435.9340330763464 - 1539.828182863033) / 1435.9340330763464 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.36 ms 2.66 ms 1.13 0.07 True
675.86 μs 736.04 μs 1.09 0.14 False
728.23 μs 802.17 μs 1.10 0.11 False
9.40 μs 10.25 μs 1.09 0.48 False
34.95 μs 42.58 μs 1.22 0.30 False
2.42 ms 2.81 ms 1.16 0.09 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CreateAddAndClear<String>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.6630716872759854 > 2.462774409926569. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -24.36073170849422 (T) = (0 -2724682.7713264334) / Math.Sqrt((2890398956.727318 / (27)) + (3229047295.6406865 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.15224531324230176 = (2364672.470360892 - 2724682.7713264334) / 2364672.470360892 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.SortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 736.0413971185066 > 704.3154990946383. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:07:16 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.409599702644424 (T) = (0 -723843.6492122879) / Math.Sqrt((117633801.596988 / (16)) + (519373410.2260402 / (40))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (40) - 2, .025) and -0.08379073229181239 = (667881.3793522934 - 723843.6492122879) / 667881.3793522934 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 802.1668747916667 > 766.8088079900568. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.101308332186955 (T) = (0 -786148.6434615239) / Math.Sqrt((239892721.83630937 / (19)) + (339460736.5139423 / (39))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (39) - 2, .025) and -0.06976779449722553 = (734877.8375133285 - 786148.6434615239) / 734877.8375133285 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.Stack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.249310791832151 > 9.972976301910677. IsChangePoint: Marked as a change because one of 8/22/2023 3:00:25 AM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -2.9096958988427555 (T) = (0 -10680.881234121052) / Math.Sqrt((864904.3009870052 / (18)) + (702233.9869424688 / (39))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (39) - 2, .025) and -0.07528734289678514 = (9933.048412294285 - 10680.881234121052) / 9933.048412294285 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.ConcurrentBag(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 42.575095398642915 > 37.043607975883454. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -7.6361045428436904 (T) = (0 -39924.10985813394) / Math.Sqrt((2319262.7666071225 / (17)) + (1846025.1826652626 / (38))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (38) - 2, .025) and -0.08964263055057137 = (36639.63646315967 - 39924.10985813394) / 36639.63646315967 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<String>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.80701515 > 2.5988927475247525. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.601122349150106 (T) = (0 -2819878.946018744) / Math.Sqrt((2404493884.741222 / (29)) + (2294848666.042277 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.1423341043242354 = (2468523.819208641 - 2819878.946018744) / 2468523.819208641 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
484.81 μs 787.29 μs 1.62 0.17 True
1.32 ms 1.83 ms 1.39 0.05 True
1.32 ms 1.78 ms 1.35 0.08 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Sort<IntStruct>.Array_Comparison(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 787.2872000166667 > 513.957184972. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -69.79093805464913 (T) = (0 -818867.1457483554) / Math.Sqrt((155599740.17210057 / (28)) + (478767502.5127368 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.6676678725536913 = (491025.31698618643 - 818867.1457483554) / 491025.31698618643 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<IntStruct>.LinqOrderByExtension(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8316561066399997 > 1.3783106076276923. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -82.03404866384695 (T) = (0 -1788596.1156357029) / Math.Sqrt((304819059.1407622 / (28)) + (647507229.0798004 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.3593023164046343 = (1315819.2214124626 - 1788596.1156357029) / 1315819.2214124626 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<IntStruct>.LinqQuery(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7795851467066666 > 1.3901291869924999. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -40.89940617324119 (T) = (0 -1798945.8357651455) / Math.Sqrt((1278564342.074299 / (29)) + (2556842169.420311 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3540231989207817 = (1328593.0678285183 - 1798945.8357651455) / 1328593.0678285183 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
477.59 μs 588.06 μs 1.23 0.11 True
1.95 ms 2.60 ms 1.33 0.10 True
608.15 μs 747.84 μs 1.23 0.12 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 588.0580353835979 > 497.42356316636034. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -32.1364609393393 (T) = (0 -594586.3284813775) / Math.Sqrt((131893835.38582814 / (28)) + (245040289.858772 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.24389117670431779 = (478005.1017459022 - 594586.3284813775) / 478005.1017459022 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.5999437499999996 > 2.066520856626154. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -60.51738890165412 (T) = (0 -2684669.344564217) / Math.Sqrt((1878173194.6918406 / (29)) + (2205610844.297428 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3651853901216601 = (1966523.6413971363 - 2684669.344564217) / 1966523.6413971363 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 747.8368471385542 > 635.7587110768169. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -33.01104472970756 (T) = (0 -766856.1305113919) / Math.Sqrt((324545213.33373195 / (29)) + (255397938.01196793 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.23839577208981363 = (619233.48560639 - 766856.1305113919) / 619233.48560639 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.80 μs 3.84 μs 1.37 0.05 True
2.60 μs 3.36 μs 1.29 0.08 True
2.43 μs 3.10 μs 1.28 0.08 True
1.74 μs 1.88 μs 1.08 0.11 False
3.70 μs 4.46 μs 1.21 0.12 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.8400156950215916 > 3.000060468584799. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.060851559612686 (T) = (0 -3659.0241405271236) / Math.Sqrt((8158.688331067496 / (28)) + (6336.984023599945 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.26614282061014544 = (2889.8984229629523 - 3659.0241405271236) / 2889.8984229629523 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.3597333840139014 > 2.7457701209231766. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -30.460669140379412 (T) = (0 -3424.3218209452425) / Math.Sqrt((5958.643475897023 / (27)) + (10229.156934024015 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.2706652138072748 = (2694.9048291681797 - 3424.3218209452425) / 2694.9048291681797 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0962799440236526 > 2.452026084614626. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -48.53704284620474 (T) = (0 -3075.189321250816) / Math.Sqrt((3643.2095683707435 / (30)) + (2221.5252246448986 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.28554138416479535 = (2392.13560849209 - 3075.189321250816) / 2392.13560849209 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8817128135584476 > 1.8067879250142131. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -10.437887078769883 (T) = (0 -1852.5438532463415) / Math.Sqrt((1732.0979255812845 / (19)) + (1107.5735297362146 / (39))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (39) - 2, .025) and -0.06565341730718795 = (1738.4112162165784 - 1852.5438532463415) / 1738.4112162165784 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.45831554710454 > 3.834562415563285. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -33.291735163406706 (T) = (0 -4568.3794553100115) / Math.Sqrt((9915.729469877293 / (29)) + (8321.377011111093 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.22821598169472296 = (3719.5245163692202 - 4568.3794553100115) / 3719.5245163692202 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
407.11 μs 584.45 μs 1.44 0.22 True
59.57 μs 78.73 μs 1.32 0.32 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IndexerSet<Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 584.4504603395061 > 427.58798101973673. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -61.94595141818944 (T) = (0 -580149.2241729867) / Math.Sqrt((77102124.63567181 / (27)) + (132607481.71276157 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.41047071650515143 = (411316.0361176969 - 580149.2241729867) / 411316.0361176969 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IndexerSet<Int32>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.72891347838787 > 63.51671526566711. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.181263596695302 (T) = (0 -69471.19646820783) / Math.Sqrt((4077910.546471699 / (18)) + (5704366.685616206 / (39))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (39) - 2, .025) and -0.13103107603196307 = (61422.889202952865 - 69471.19646820783) / 61422.889202952865 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<StructRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.55 μs 4.49 μs 1.27 0.10 True
3.11 μs 3.95 μs 1.27 0.10 True
3.43 μs 4.21 μs 1.23 0.09 True
4.79 μs 6.03 μs 1.26 0.08 True
2.43 μs 2.81 μs 1.16 0.16 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.493486170675166 > 3.745053923723155. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.82507456662603 (T) = (0 -4508.044515487241) / Math.Sqrt((9446.247170547336 / (28)) + (8049.036637127567 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.23695555581467884 = (3644.467656332382 - 4508.044515487241) / 3644.467656332382 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.952666690263961 > 3.244169746938734. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -38.97388371165015 (T) = (0 -3954.3244837418356) / Math.Sqrt((7727.813106150606 / (29)) + (5689.141159998015 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2690304161337713 = (3116.0202572520543 - 3954.3244837418356) / 3116.0202572520543 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.20662338386215 > 3.5163759436718833. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -46.735274503964526 (T) = (0 -4266.065140850899) / Math.Sqrt((6325.83142641728 / (29)) + (3686.579408536795 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2555838205441505 = (3397.674508900611 - 4266.065140850899) / 3397.674508900611 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.030829860919 > 5.024413365452726. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -31.901745673537746 (T) = (0 -5995.131492741415) / Math.Sqrt((16998.77819133002 / (29)) + (19249.914649395654 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.23172748227375933 = (4867.254793791277 - 5995.131492741415) / 4867.254793791277 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.8142190456271883 > 2.5605977509407274. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -14.51303440135378 (T) = (0 -2852.485262111022) / Math.Sqrt((7661.619486633608 / (28)) + (7066.386908712089 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.13086240236089675 = (2522.3981769629095 - 2852.485262111022) / 2522.3981769629095 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.01 μs 10.83 μs 1.20 0.10 True
5.57 μs 6.64 μs 1.19 0.13 True
4.26 μs 5.10 μs 1.20 0.19 True
7.07 μs 7.77 μs 1.10 0.15 False
8.61 μs 9.60 μs 1.12 0.13 True
8.49 μs 10.28 μs 1.21 0.11 True
236.35 μs 260.89 μs 1.10 0.03 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCachedCustomConverter #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.831775145404729 > 9.961187094330604. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -22.6347462128331 (T) = (0 -11084.595971933162) / Math.Sqrt((81715.45159036233 / (29)) + (44202.33175758566 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.15659884005581237 = (9583.786173776785 - 11084.595971933162) / 9583.786173776785 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedDefaultOptions #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.644401511739627 > 5.723478961805356. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -32.822533470053955 (T) = (0 -6812.862032062995) / Math.Sqrt((24141.382166620057 / (29)) + (23763.143753103475 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.24348508327123963 = (5478.845000810448 - 6812.862032062995) / 5478.845000810448 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedJsonSerializerContext #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.101978989698704 > 4.415839447250023. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -23.800629932089805 (T) = (0 -4997.440658647348) / Math.Sqrt((14307.938668061925 / (29)) + (12906.170051099134 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.1708155236430884 = (4268.3416454006365 - 4997.440658647348) / 4268.3416454006365 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewJsonSerializerContext #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.772159480480736 > 7.549964469737836. IsChangePoint: Marked as a change because one of 8/22/2023 10:50:09 AM, 9/25/2023 2:54:53 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.767488828069224 (T) = (0 -7793.623742490522) / Math.Sqrt((22672.402622056088 / (20)) + (49559.83938198563 / (36))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (36) - 2, .025) and -0.08184062358586748 = (7204.040569911109 - 7793.623742490522) / 7204.040569911109 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewDefaultOptions #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.598367957467852 > 8.86275646291401. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.80809159954689 (T) = (0 -9534.653009239666) / Math.Sqrt((60629.43612552667 / (28)) + (35413.925766409506 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.13752248552604845 = (8381.946845499371 - 9534.653009239666) / 8381.946845499371 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomizedOptions #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.282797430332261 > 8.961339961317023. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.378702976096697 (T) = (0 -9924.600055988456) / Math.Sqrt((45300.91682000731 / (29)) + (64151.33273559962 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.13630791349145652 = (8734.076334550737 - 9924.600055988456) / 8734.076334550737 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomConverter #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 260.89090270833333 > 249.4213871027166. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -15.2551213785763 (T) = (0 -257218.3065924132) / Math.Sqrt((18405257.301541835 / (24)) + (20823221.60653489 / (33))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (33) - 2, .025) and -0.07540921574174918 = (239181.7950109348 - 257218.3065924132) / 239181.7950109348 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
159.33 μs 191.69 μs 1.20 0.08 True
94.11 μs 111.31 μs 1.18 0.03 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 191.69256771799627 > 165.48336397948955. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -23.95131634869615 (T) = (0 -187854.47854729925) / Math.Sqrt((23549651.659232147 / (28)) + (10902328.829751303 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.16367619298447764 = (161431.91695407062 - 187854.47854729925) / 161431.91695407062 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 111.30599296362551 > 98.00538840899735. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -33.95026159293076 (T) = (0 -111536.8472479512) / Math.Sqrt((3411003.125705619 / (27)) + (3118423.523124195 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.17260780514362684 = (95118.62939910212 - 111536.8472479512) / 95118.62939910212 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
563.60 μs 1.03 ms 1.82 0.21 True
88.82 μs 178.48 μs 2.01 0.38 True
649.73 μs 1.20 ms 1.85 0.29 True
191.61 μs 284.21 μs 1.48 0.11 True
86.08 μs 120.32 μs 1.40 0.28 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrueComparer<Int32>.SortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0281448097098216 > 600.3648804533229. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -71.9607298727396 (T) = (0 -1022630.4206740883) / Math.Sqrt((580092880.1155154 / (28)) + (461160090.77458006 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.7416380361301815 = (587165.874573062 - 1022630.4206740883) / 587165.874573062 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<Int32>.HashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 178.47752823033704 > 94.07197190818476. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -57.89642696577117 (T) = (0 -176493.48107021285) / Math.Sqrt((23398711.22559817 / (28)) + (35957985.67082208 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.896098277032599 = (93082.45422089925 - 176493.48107021285) / 93082.45422089925 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<Int32>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2031294074519232 > 721.5786559618432. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -57.5530815080907 (T) = (0 -1247989.77750743) / Math.Sqrt((966442145.4051126 / (29)) + (1634671425.0835774 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.7754273595284079 = (702923.5923450696 - 1247989.77750743) / 702923.5923450696 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<Int32>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 284.20997999043374 > 202.4397841936729. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -73.00247283258031 (T) = (0 -283492.582923826) / Math.Sqrt((23302714.46063401 / (28)) + (17550660.637388196 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.4466293359710512 = (195967.67179723433 - 283492.582923826) / 195967.67179723433 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<Int32>.FrozenSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 120.31508397932816 > 88.26631886917582. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.24362877960519 (T) = (0 -119168.6013219558) / Math.Sqrt((8527659.75748112 / (28)) + (14257763.189337594 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.41612214894517513 = (84151.35757231164 - 119168.6013219558) / 84151.35757231164 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Activator<EmptyClass>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.93 μs 17.21 μs 1.44 0.44 False
432.79 ns 595.48 ns 1.38 0.17 True
299.19 ns 445.47 ns 1.49 0.18 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Activator&lt;EmptyClass&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Reflection.Activator<EmptyClass>.CreateInstanceNames #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.207716916944754 > 12.790621764527412. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -6.769859542139685 (T) = (0 -14320.619740222375) / Math.Sqrt((902020.4772294882 / (27)) + (643098.7036071185 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.1254331670544932 = (12724.540345387719 - 14320.619740222375) / 12724.540345387719 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Activator<EmptyClass>.CreateInstanceType #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 595.4786075318206 > 452.00861053260127. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -47.15534538220577 (T) = (0 -576.7105069293511) / Math.Sqrt((123.5180119092498 / (29)) + (136.36430341992462 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.32410413594780385 = (435.5476969464621 - 576.7105069293511) / 435.5476969464621 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Activator<EmptyClass>.CreateInstanceGeneric #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 445.47263546898677 > 321.82955110568315. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -63.439625962315695 (T) = (0 -448.3308316905235) / Math.Sqrt((85.39208176926591 / (28)) + (49.56805127531029 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.4465223460325735 = (309.937024422869 - 448.3308316905235) / 309.937024422869 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
86.33 μs 98.62 μs 1.14 0.05 True
93.36 μs 110.12 μs 1.18 0.03 True
88.78 μs 104.92 μs 1.18 0.05 True
92.97 μs 111.41 μs 1.20 0.04 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 98.62465764767933 > 87.66457514070008. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -30.65149071337755 (T) = (0 -101620.54280450166) / Math.Sqrt((5541890.28360147 / (30)) + (2932355.6550331125 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.19226935373672008 = (85232.87333186102 - 101620.54280450166) / 85232.87333186102 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 110.12350349178404 > 97.89602603154806. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -43.49883879169112 (T) = (0 -110547.43901905042) / Math.Sqrt((2139789.7389512057 / (28)) + (1598156.7498240878 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.16646054032043442 = (94771.69196712159 - 110547.43901905042) / 94771.69196712159 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 104.91694642058165 > 92.07996913788641. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -40.72365281648652 (T) = (0 -106438.72785355101) / Math.Sqrt((2967855.02035303 / (27)) + (1840513.764792959 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.18950518174638917 = (89481.51675748185 - 106438.72785355101) / 89481.51675748185 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 111.40975800382652 > 99.11243947003292. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.68906459892246 (T) = (0 -111411.60565963558) / Math.Sqrt((2663939.1995796696 / (29)) + (1763168.1750482847 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.16445081936420042 = (95677.38182404923 - 111411.60565963558) / 95677.38182404923 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.34 μs 4.05 μs 1.21 0.15 True
1.59 μs 1.98 μs 1.25 0.10 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.0507744310958556 > 3.329385626479815. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.173708390551937 (T) = (0 -3768.5865096455955) / Math.Sqrt((10500.159739943718 / (29)) + (12236.602069439832 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.16612635963973094 = (3231.7136805053287 - 3768.5865096455955) / 3231.7136805053287 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.980128617117153 > 1.646707773874147. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -26.906352550903446 (T) = (0 -1957.6379748520878) / Math.Sqrt((1925.8020112170477 / (28)) + (3140.5262727924314 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.22383062716261376 = (1599.5987773167333 - 1957.6379748520878) / 1599.5987773167333 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
76.88 μs 92.76 μs 1.21 0.04 True
87.48 μs 103.67 μs 1.19 0.03 True
143.17 μs 179.83 μs 1.26 0.05 True
251.55 μs 310.79 μs 1.24 0.06 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 92.75515875496033 > 82.76898606101906. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -29.55935668309835 (T) = (0 -92510.92809911164) / Math.Sqrt((2933398.8413679423 / (29)) + (2217226.2110052914 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.15561277328059817 = (80053.5700522659 - 92510.92809911164) / 80053.5700522659 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 103.66993407468422 > 87.97302731522896. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.12395101038323 (T) = (0 -101793.2928552247) / Math.Sqrt((3496412.2014180366 / (27)) + (2122001.115872942 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.19013714497296966 = (85530.72499685457 - 101793.2928552247) / 85530.72499685457 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 179.8286222734255 > 152.80499665198448. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -32.16172986524642 (T) = (0 -175861.24198725008) / Math.Sqrt((11111434.865997195 / (27)) + (9511280.50852598 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.1866173046123701 = (148203.84070220374 - 175861.24198725008) / 148203.84070220374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 310.7940449513266 > 264.3532995035262. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -28.801136745931935 (T) = (0 -309799.6122124209) / Math.Sqrt((70259292.96866862 / (29)) + (21930474.111882783 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.19869109230166357 = (258448.24759443235 - 309799.6122124209) / 258448.24759443235 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
199.66 ns 266.81 ns 1.34 0.23 True
357.28 ns 442.43 ns 1.24 0.20 True
172.79 ns 257.12 ns 1.49 0.29 True
313.54 ns 409.08 ns 1.30 0.20 True
807.41 ns 885.30 ns 1.10 0.09 True
155.45 ns 228.98 ns 1.47 0.21 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'
### Payloads [Baseline]() [Compare]() ### System.Reflection.Invoke.Property_Get_int #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 266.81160234680567 > 208.89686343750003. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -26.852912100011764 (T) = (0 -273.4352144641339) / Math.Sqrt((40.343225219187545 / (29)) + (170.7961750670673 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.360517917955284 = (200.97876761158602 - 273.4352144641339) / 200.97876761158602 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Invoke.Ctor0_ActivatorCreateInstance_NoParams #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 442.4294645408163 > 370.3516579246795. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -24.316984819187063 (T) = (0 -442.26159254263837) / Math.Sqrt((135.40142209100267 / (28)) + (211.66884067109 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.23690970324012628 = (357.5536608566651 - 442.26159254263837) / 357.5536608566651 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Invoke.Property_Get_class #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 257.1186486338798 > 177.660339788048. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -45.92468063970807 (T) = (0 -239.2582959218561) / Math.Sqrt((21.075285274340512 / (28)) + (42.589280202543875 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.40070908737756056 = (170.81226792766864 - 239.2582959218561) / 170.81226792766864 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Invoke.Method0_NoParms #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 409.07635061813187 > 327.6473205294118. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -33.909412101336706 (T) = (0 -393.2352522569371) / Math.Sqrt((74.2315206808228 / (29)) + (88.68211483504946 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2568882210903406 = (312.86413991198725 - 393.2352522569371) / 312.86413991198725 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Invoke.StaticMethod4_ByRefParams_int_string_struct_class #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 885.3023493589743 > 855.4374791899671. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -20.84605849365963 (T) = (0 -913.0136955054854) / Math.Sqrt((264.0296242092305 / (29)) + (367.2181124990366 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.11922427771522162 = (815.7557995161673 - 913.0136955054854) / 815.7557995161673 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Invoke.Ctor0_NoParams #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 228.9771525735294 > 163.58391875000004. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -54.99001597619227 (T) = (0 -233.42519180924842) / Math.Sqrt((21.73188179657488 / (28)) + (29.942716052268104 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.4637435969104685 = (159.47136663958102 - 233.42519180924842) / 159.47136663958102 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
109.85 μs 139.64 μs 1.27 0.11 True
120.25 μs 164.07 μs 1.36 0.07 True
124.08 μs 164.38 μs 1.32 0.08 True
125.10 μs 166.60 μs 1.33 0.08 True
99.70 μs 144.04 μs 1.44 0.11 True
122.77 μs 167.98 μs 1.37 0.12 True
106.28 μs 141.87 μs 1.33 0.12 True
97.69 μs 136.27 μs 1.39 0.11 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 139.64240357669618 > 113.838745830256. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.940338874095524 (T) = (0 -145253.17448870448) / Math.Sqrt((8480057.14377578 / (28)) + (17431385.263032295 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.3303877710566861 = (109181.08062082858 - 145253.17448870448) / 109181.08062082858 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 164.0700903846154 > 127.16138460886876. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -47.39974198922947 (T) = (0 -166507.91246440733) / Math.Sqrt((15548850.749390077 / (29)) + (9293853.581163593 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.35772954102984744 = (122637.02558766594 - 166507.91246440733) / 122637.02558766594 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 164.37633832275972 > 125.4589023072152. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -53.81494207536341 (T) = (0 -164287.14211828535) / Math.Sqrt((8808804.605415266 / (29)) + (9160444.321756596 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.34743448641063024 = (121925.88491327874 - 164287.14211828535) / 121925.88491327874 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 166.6002209219858 > 131.00872899049537. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -48.368506999420475 (T) = (0 -168939.05420645326) / Math.Sqrt((12526083.430781877 / (28)) + (8075067.641905829 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.3226083227982528 = (127731.73379782427 - 168939.05420645326) / 127731.73379782427 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 144.036855530303 > 104.14933699777411. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -53.11101843266703 (T) = (0 -138759.07754904538) / Math.Sqrt((8651603.439970722 / (28)) + (5264426.560414925 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.366253626217279 = (101561.72681731507 - 138759.07754904538) / 101561.72681731507 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 167.98284960612196 > 129.06827805822311. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -47.62198702731449 (T) = (0 -170321.19196810442) / Math.Sqrt((19603081.19698401 / (29)) + (4513020.4965385655 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.34223237251195043 = (126893.96818030329 - 170321.19196810442) / 126893.96818030329 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 141.86794427614797 > 111.5096874017258. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -52.19167817686669 (T) = (0 -144104.71022326723) / Math.Sqrt((8008418.921064367 / (29)) + (6051250.8623441 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.34068272490968277 = (107486.06478313135 - 144104.71022326723) / 107486.06478313135 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 136.26725146198828 > 103.19511939749955. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -45.38641166274704 (T) = (0 -136959.67594386748) / Math.Sqrt((10823671.319291478 / (29)) + (7126742.759679889 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.3560427463999241 = (100999.52697470153 - 136959.67594386748) / 100999.52697470153 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Logging.ScopesOverheadBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
434.69 ns 477.75 ns 1.10 0.20 False
621.62 ns 724.32 ns 1.17 0.10 False
414.69 ns 465.39 ns 1.12 0.21 False
366.38 ns 406.86 ns 1.11 0.16 False
437.66 ns 472.43 ns 1.08 0.13 False
371.90 ns 409.56 ns 1.10 0.16 False
375.04 ns 418.00 ns 1.11 0.17 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.ScopesOverheadBenchmark*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered_InsideScope(HasISupportLoggingScopeLogger: False, CaptureScopes: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 477.75134929010983 > 453.9677059231391. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -10.7712389674073 (T) = (0 -472.91318598489516) / Math.Sqrt((154.77449295591236 / (29)) + (154.81137907245832 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.08117366046783936 = (437.40723925910186 - 472.91318598489516) / 437.40723925910186 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered_InsideScope(HasISupportLoggingScopeLogger: True, CaptureScopes: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 724.3187711337059 > 656.6479558002462. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.16156361164893 (T) = (0 -681.0632193692117) / Math.Sqrt((284.29822627146905 / (19)) + (317.98639849851554 / (39))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (39) - 2, .025) and -0.08553482986413245 = (627.3987721374677 - 681.0632193692117) / 627.3987721374677 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered_InsideScope(HasISupportLoggingScopeLogger: False, CaptureScopes: False) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 465.38648387597544 > 445.60260169233726. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.657391623304456 (T) = (0 -469.4564979065186) / Math.Sqrt((125.41105090207017 / (28)) + (152.9763978762757 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.0919480524797263 = (429.92566985253615 - 469.4564979065186) / 429.92566985253615 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered(HasISupportLoggingScopeLogger: True, CaptureScopes: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 406.86218465310174 > 398.41334257329277. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.8432429441317 (T) = (0 -411.8126197130496) / Math.Sqrt((111.90675380155287 / (29)) + (71.1444611860293 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.08501553920636426 = (379.5453657873604 - 411.8126197130496) / 379.5453657873604 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered_InsideScope(HasISupportLoggingScopeLogger: True, CaptureScopes: False) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 472.43190705772065 > 458.03813823148226. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.285549536808523 (T) = (0 -470.372773590028) / Math.Sqrt((151.9767043687195 / (29)) + (151.44841001892945 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.08413793312493686 = (433.868015515533 - 470.372773590028) / 433.868015515533 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered(HasISupportLoggingScopeLogger: False, CaptureScopes: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 409.55745738829745 > 395.37187284203713. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -15.030696908813256 (T) = (0 -414.84517952665277) / Math.Sqrt((67.28666510921786 / (29)) + (89.80231158707115 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.0920929834526954 = (379.862507874653 - 414.84517952665277) / 379.862507874653 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered(HasISupportLoggingScopeLogger: True, CaptureScopes: False) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 417.99668135319075 > 395.80887805946367. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -15.407544865250301 (T) = (0 -414.6372270056104) / Math.Sqrt((89.75180587496334 / (28)) + (58.81482579351753 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.09290625800076674 = (379.38956243520704 - 414.6372270056104) / 379.38956243520704 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
26.28 μs 29.55 μs 1.12 0.08 True
9.04 μs 11.09 μs 1.23 0.15 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<Location>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.55128431339455 > 26.13222022553647. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.89186062388696 (T) = (0 -29338.00917732579) / Math.Sqrt((741329.2118096104 / (29)) + (278124.80280634685 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.16266384040374446 = (25233.44079156872 - 29338.00917732579) / 25233.44079156872 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<Location>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.09035287747525 > 9.474580475611383. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -32.03543409245195 (T) = (0 -11183.611464525538) / Math.Sqrt((70520.7280219636 / (29)) + (43196.25535243109 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.21858357760372896 = (9177.549796393478 - 11183.611464525538) / 9177.549796393478 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.42 μs 1.73 μs 1.22 0.11 True
2.40 μs 3.07 μs 1.28 0.11 True
1.22 μs 1.50 μs 1.23 0.15 True
1.64 μs 1.86 μs 1.14 0.18 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7258784212688103 > 1.4836216578884251. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -29.7722250612905 (T) = (0 -1731.007627050553) / Math.Sqrt((1626.102128959425 / (29)) + (1401.5630972153165 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.21320740948399275 = (1426.802716105067 - 1731.007627050553) / 1426.802716105067 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.065960622182445 > 2.437709984207554. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -24.804960245311307 (T) = (0 -2806.3734295765908) / Math.Sqrt((5289.63160026465 / (29)) + (5496.193253116971 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.205486394224701 = (2328.000915664824 - 2806.3734295765908) / 2328.000915664824 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4969811835933757 > 1.275464160958821. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -31.200338160267876 (T) = (0 -1486.4470023482777) / Math.Sqrt((1245.0350349510632 / (29)) + (762.5705848350107 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2115962663939056 = (1226.8501014553428 - 1486.4470023482777) / 1226.8501014553428 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8613269798028182 > 1.6452347085857295. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.74282432608345 (T) = (0 -1890.945285165093) / Math.Sqrt((1644.376233454929 / (28)) + (1466.1705053484718 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.1811595343469905 = (1600.9228475732627 - 1890.945285165093) / 1600.9228475732627 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
207.24 μs 247.08 μs 1.19 0.08 False
91.50 μs 107.15 μs 1.17 0.05 True
106.43 μs 127.35 μs 1.20 0.07 True
1.72 ms 1.87 ms 1.09 0.01 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 247.08458303455015 > 223.9180692274763. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.39381194801734 (T) = (0 -244208.73071763667) / Math.Sqrt((34820714.19751556 / (30)) + (19569565.830772072 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.1205635555183983 = (217933.85079765524 - 244208.73071763667) / 217933.85079765524 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 107.15079874429223 > 96.44373581273649. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -30.05935944819837 (T) = (0 -108303.04560167625) / Math.Sqrt((4417826.998581278 / (29)) + (2882311.722510567 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.16310407234003232 = (93115.52437761043 - 108303.04560167625) / 93115.52437761043 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 127.34823790650405 > 108.61553692223072. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -30.54151154130886 (T) = (0 -126509.24765065456) / Math.Sqrt((10546478.188144019 / (29)) + (4463171.450484141 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.21018768198860108 = (104536.88261210229 - 126509.24765065456) / 104536.88261210229 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8704829948979595 > 1.8193120887407408. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.35148974773797 (T) = (0 -1864481.4674194078) / Math.Sqrt((622091307.0011339 / (28)) + (404529126.5627263 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.06657365306757834 = (1748103.8107935279 - 1864481.4674194078) / 1748103.8107935279 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.82 μs 6.77 μs 1.16 0.11 True
3.45 μs 4.30 μs 1.25 0.04 True
7.69 μs 8.55 μs 1.11 0.07 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<Location>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.771624226050361 > 6.056201252090844. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.191196919025337 (T) = (0 -7037.298634775841) / Math.Sqrt((56861.170096369126 / (28)) + (33064.74558716919 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.20419151530913435 = (5844.002839506188 - 7037.298634775841) / 5844.002839506188 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<Location>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.298876656250309 > 3.5416593543290595. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.33297503855339 (T) = (0 -4177.337502087752) / Math.Sqrt((6571.728734161982 / (29)) + (4123.375171985971 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.22073557852612505 = (3421.98390509051 - 4177.337502087752) / 3421.98390509051 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<Location>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.546701892608056 > 7.87759681786959. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.19566431074839 (T) = (0 -8699.316281458076) / Math.Sqrt((41893.13318310507 / (28)) + (29697.506523886557 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.1393619325511592 = (7635.2527084868725 - 8699.316281458076) / 7635.2527084868725 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
89.69 μs 107.82 μs 1.20 0.05 True
120.96 μs 145.60 μs 1.20 0.05 True
86.87 μs 103.25 μs 1.19 0.02 True
112.00 μs 131.73 μs 1.18 0.07 True
102.20 μs 124.13 μs 1.21 0.06 True
97.14 μs 120.89 μs 1.24 0.04 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 107.818806706621 > 96.49664364309214. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -24.97929766440501 (T) = (0 -108515.81497839271) / Math.Sqrt((5336746.193426902 / (27)) + (5531801.00858337 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.16748825757848007 = (92948.09971234176 - 108515.81497839271) / 92948.09971234176 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 145.59552466750537 > 126.75300831718349. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -28.146251883233433 (T) = (0 -146378.36650025623) / Math.Sqrt((10833642.338490946 / (27)) + (7427113.914469645 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.18469225097992656 = (123558.1361988131 - 146378.36650025623) / 123558.1361988131 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 103.2471510808271 > 93.42230856665465. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -32.11912719622337 (T) = (0 -104517.2271901509) / Math.Sqrt((3188739.681910006 / (29)) + (2772477.49833215 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.1618853743677668 = (89954.85225642275 - 104517.2271901509) / 89954.85225642275 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 131.73177064732144 > 117.73048421875001. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -20.78852677780462 (T) = (0 -129429.46045414543) / Math.Sqrt((11563428.845560232 / (29)) + (5520626.472957774 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.14061296346077024 = (113473.60112534525 - 129429.46045414543) / 113473.60112534525 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 124.12755084325399 > 106.96468952246259. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.292158140103986 (T) = (0 -125817.5190788334) / Math.Sqrt((5455300.855872075 / (27)) + (3758593.6785274246 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.21928761875366073 = (103189.36823736664 - 125817.5190788334) / 103189.36823736664 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 120.88777486263737 > 102.44387054667664. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.835745056784695 (T) = (0 -121457.52362573749) / Math.Sqrt((7716803.664064199 / (28)) + (3838407.090929579 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2242794600750626 = (99207.3522317288 - 121457.52362573749) / 99207.3522317288 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.06 μs 3.76 μs 1.23 0.12 True
3.85 μs 4.38 μs 1.14 0.07 False
3.77 μs 4.13 μs 1.10 0.07 False
2.60 μs 2.96 μs 1.14 0.12 False
3.06 μs 3.45 μs 1.13 0.12 False
2.81 μs 3.32 μs 1.18 0.11 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ClassRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.7568746246176232 > 3.3088692186083573. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -17.886575634609258 (T) = (0 -3762.110041609563) / Math.Sqrt((15999.663431100487 / (30)) + (6216.509024483973 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.14942985268968081 = (3273.022736277622 - 3762.110041609563) / 3273.022736277622 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.375695977697623 > 4.143954051309472. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.821868318602778 (T) = (0 -4452.11135358596) / Math.Sqrt((8077.359437542225 / (29)) + (6669.078688059959 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.11366680522467935 = (3997.70499820883 - 4452.11135358596) / 3997.70499820883 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.1284936844934155 > 3.965455693068941. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -14.567848690276369 (T) = (0 -4132.130230907709) / Math.Sqrt((8803.785486727062 / (29)) + (4708.461910914683 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.0823685418669997 = (3817.6739909496196 - 4132.130230907709) / 3817.6739909496196 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.962140954900479 > 2.7279363265123564. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -14.21936934165268 (T) = (0 -2958.3405409843394) / Math.Sqrt((6852.452804050733 / (29)) + (3646.4350372809595 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.10066027466410667 = (2687.7871483888607 - 2958.3405409843394) / 2687.7871483888607 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.447482348569305 > 3.194020838169308. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -15.354690335038264 (T) = (0 -3447.006591023132) / Math.Sqrt((7256.3506762949955 / (29)) + (6186.285450303877 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.10607870305705039 = (3116.420722590605 - 3447.006591023132) / 3116.420722590605 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.3188758586832328 > 2.9119568192365586. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -17.331297201159682 (T) = (0 -3287.4754319397384) / Math.Sqrt((13083.388784420807 / (28)) + (5407.101470841955 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.15579097575505682 = (2844.3511853794253 - 3287.4754319397384) / 2844.3511853794253 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
854.00 ns 1.17 μs 1.37 0.13 True
864.86 ns 1.15 μs 1.33 0.09 True
6.15 μs 6.66 μs 1.08 0.06 False
11.18 μs 12.62 μs 1.13 0.07 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: False) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.166878772682714 > 927.0632793287164. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -38.1003212050523 (T) = (0 -1162.090766626097) / Math.Sqrt((627.1311787822923 / (29)) + (772.7399241525105 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.29874157571859394 = (894.7821401521793 - 1162.090766626097) / 894.7821401521793 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.152615211555544 > 928.5389469553453. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.46246377309231 (T) = (0 -1156.7479784648087) / Math.Sqrt((499.5669656487823 / (29)) + (840.8554197351441 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2823485059494842 = (902.0542957690916 - 1156.7479784648087) / 902.0542957690916 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: True, Json: False) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.660994608912552 > 6.548374492547765. IsChangePoint: Marked as a change because one of 9/26/2023 6:21:18 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.66919589955708 (T) = (0 -6908.368568541769) / Math.Sqrt((12068.776312615411 / (27)) + (16083.43173429304 / (31))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (31) - 2, .025) and -0.10800816195224015 = (6234.943753816453 - 6908.368568541769) / 6234.943753816453 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: True, Json: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.624758206779077 > 11.730159864073823. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -17.63646940159101 (T) = (0 -12756.905796098603) / Math.Sqrt((106770.04218373555 / (29)) + (76014.06127902663 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.12329017644275872 = (11356.73227063842 - 12756.905796098603) / 11356.73227063842 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.16 ms 3.80 ms 1.20 0.14 True
9.33 ms 10.96 ms 1.17 0.02 True
3.93 ms 5.44 ms 1.38 0.09 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.797072820512821 > 3.330518489429716. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -22.49805932033069 (T) = (0 -3797423.410892846) / Math.Sqrt((14429636205.077911 / (29)) + (4204313118.49526 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.17671852631070337 = (3227129.790162041 - 3797423.410892846) / 3227129.790162041 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.96098376948052 > 9.839509053149998. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -32.18238365097251 (T) = (0 -10823596.808990447) / Math.Sqrt((26005961864.90953 / (27)) + (18615220269.099125 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.1352324130575595 = (9534256.320112364 - 10823596.808990447) / 9534256.320112364 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.444979259259259 > 4.110627787459016. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.54582609150712 (T) = (0 -5314160.362232686) / Math.Sqrt((20302314451.07406 / (28)) + (12544439665.841965 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.3054608281773974 = (4070716.062504904 - 5314160.362232686) / 4070716.062504904 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.29 ms 2.80 ms 1.22 0.08 True
1.55 ms 1.88 ms 1.21 0.04 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.8031639602272724 > 2.3839931824545455. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -24.19763790887482 (T) = (0 -2798405.285333515) / Math.Sqrt((7007758690.696911 / (27)) + (2899050383.247967 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.19610544668524607 = (2339597.4770357455 - 2798405.285333515) / 2339597.4770357455 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8824266289940828 > 1.6268064521492627. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -38.65047001791488 (T) = (0 -1872913.6869761802) / Math.Sqrt((797575656.681699 / (30)) + (1086060378.5003412 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.19780428706531122 = (1563622.4608653935 - 1872913.6869761802) / 1563622.4608653935 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.35 μs 10.30 μs 1.23 0.09 True
8.94 μs 10.94 μs 1.22 0.05 True
7.82 μs 10.32 μs 1.32 0.09 True
7.48 μs 9.61 μs 1.28 0.12 True
7.72 μs 9.34 μs 1.21 0.11 True
9.10 μs 11.22 μs 1.23 0.06 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.296580648012526 > 8.685416201300448. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.54157345730693 (T) = (0 -9926.08345037205) / Math.Sqrt((37782.07614822331 / (28)) + (29056.337863866334 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.20299327569576478 = (8251.154558308888 - 9926.08345037205) / 8251.154558308888 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.940726433944521 > 9.296337516261616. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -45.14573012230641 (T) = (0 -11099.655414147706) / Math.Sqrt((28398.894473523735 / (28)) + (38255.385545140416 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.24451231902315493 = (8918.87950362763 - 11099.655414147706) / 8918.87950362763 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.320274059184353 > 8.260226722529518. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.11755127498265 (T) = (0 -10158.604577500233) / Math.Sqrt((36412.40065877594 / (29)) + (48985.31309896214 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.26415747510604365 = (8035.869563361226 - 10158.604577500233) / 8035.869563361226 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.606857951770452 > 7.735811857935945. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.91185792979149 (T) = (0 -9445.142405994238) / Math.Sqrt((32551.69773551724 / (29)) + (45477.26241906734 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2629564621975285 = (7478.596997365854 - 9445.142405994238) / 7478.596997365854 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.34200915553907 > 7.881860325974394. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.53020781594906 (T) = (0 -9347.354506713164) / Math.Sqrt((26327.714279226555 / (27)) + (37227.07148703007 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.21295827639611933 = (7706.245704086008 - 9347.354506713164) / 7706.245704086008 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.220502157685287 > 9.379879231645981. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -30.404023139014527 (T) = (0 -10986.33006924597) / Math.Sqrt((72031.54047179491 / (28)) + (33276.50141933262 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.20306786889251552 = (9131.9287575683 - 10986.33006924597) / 9131.9287575683 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.ScopeValidation

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.40 μs 3.51 μs 1.46 0.08 True
2.33 μs 3.46 μs 1.48 0.06 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ScopeValidation*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.ScopeValidation.TransientWithScopeValidation #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.5134334270505327 > 2.5432730277748647. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -67.12470000525178 (T) = (0 -3570.01967363022) / Math.Sqrt((3853.374211733245 / (27)) + (3422.682684791322 / (28))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (28) - 2, .025) and -0.44103542433543885 = (2477.3989683679033 - 3570.01967363022) / 2477.3989683679033 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ScopeValidation.Transient #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.4597249787744557 > 2.493117770445168. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -58.11829937720835 (T) = (0 -3496.8761191835474) / Math.Sqrt((5758.421440162453 / (28)) + (3953.2000153524327 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.44372574351268657 = (2422.1193913709685 - 3496.8761191835474) / 2422.1193913709685 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
855.82 ns 1.00 μs 1.17 0.17 False
864.68 ns 1.04 μs 1.20 0.31 False
1.32 μs 1.47 μs 1.11 0.18 False
1.15 μs 1.37 μs 1.20 0.14 False
1.32 μs 1.43 μs 1.08 0.21 False
1.13 μs 1.33 μs 1.17 0.20 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0024676871726075 > 906.0826485615727. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.874949149897004 (T) = (0 -992.2720967471722) / Math.Sqrt((1194.1741359799619 / (27)) + (693.1902595054252 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.11994629004950422 = (885.9997176322728 - 992.2720967471722) / 885.9997176322728 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.041564731013373 > 914.1133353159348. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -16.018385583938453 (T) = (0 -1003.9402498650114) / Math.Sqrt((699.1071630124782 / (27)) + (950.7461043970549 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.13923883191362948 = (881.2377367602971 - 1003.9402498650114) / 881.2377367602971 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4694537726240566 > 1.3938375328076125. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.182359484785783 (T) = (0 -1472.1198808895278) / Math.Sqrt((1550.6655519630328 / (28)) + (1760.6012682635894 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.09789212157809951 = (1340.8602283924918 - 1472.1198808895278) / 1340.8602283924918 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.374491488991522 > 1.2338200497305878. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.774385133154697 (T) = (0 -1333.9250217456201) / Math.Sqrt((2308.2068899366145 / (29)) + (1505.17749139616 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.12336302312056475 = (1187.4389616636481 - 1333.9250217456201) / 1187.4389616636481 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4285819092486791 > 1.3638977188380366. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.544159263786167 (T) = (0 -1471.6782794613343) / Math.Sqrt((1986.87307071296 / (29)) + (1177.0532540968884 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.10635227571352635 = (1330.2076669135029 - 1471.6782794613343) / 1330.2076669135029 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.327224653250028 > 1.2120108188632923. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -10.780331699002856 (T) = (0 -1313.0393260678861) / Math.Sqrt((1849.0262152539653 / (28)) + (2662.786661437212 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.11501865403206402 = (1177.5940441173352 - 1313.0393260678861) / 1177.5940441173352 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.00 μs 1.14 μs 1.14 0.10 True
970.34 ns 1.06 μs 1.09 0.09 True
2.20 μs 2.40 μs 1.09 0.09 False
2.53 μs 2.78 μs 1.10 0.05 False
1.51 μs 1.69 μs 1.12 0.20 False
1.44 μs 1.63 μs 1.14 0.24 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1414979005047492 > 1037.9117935511829. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -16.262123240385055 (T) = (0 -1122.5600476552927) / Math.Sqrt((710.8154108304475 / (28)) + (892.5729426231347 / (28))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (28) - 2, .025) and -0.12312177266709133 = (999.499853866723 - 1122.5600476552927) / 999.499853866723 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0590084211519977 > 1007.3943100277656. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -16.01737396696466 (T) = (0 -1085.9324757983873) / Math.Sqrt((978.5634623948715 / (29)) + (430.45296731328926 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.11459468982261223 = (974.2846307398194 - 1085.9324757983873) / 974.2846307398194 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: BasicJson) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3998836575453657 > 2.3367740516961906. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.07217049771069 (T) = (0 -2452.437369138227) / Math.Sqrt((2142.152580951558 / (30)) + (3127.291317319715 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.07684654710296437 = (2277.4251129244076 - 2452.437369138227) / 2277.4251129244076 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.7809344011468027 > 2.6799697351181537. IsChangePoint: Marked as a change because one of 9/26/2023 10:37:55 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.950812849955057 (T) = (0 -2718.7768644727926) / Math.Sqrt((1820.921775037803 / (28)) + (3181.396172439714 / (30))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (30) - 2, .025) and -0.061001246397576696 = (2562.463403039224 - 2718.7768644727926) / 2562.463403039224 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6912473488117068 > 1.5628957555158962. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.233352767270935 (T) = (0 -1671.1508194027006) / Math.Sqrt((3132.4948173104285 / (29)) + (1287.7516736321938 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.10835732759010341 = (1507.7726088898394 - 1671.1508194027006) / 1507.7726088898394 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6307502558877491 > 1.563229853301453. IsChangePoint: Marked as a change because one of 9/26/2023 10:37:55 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -9.956826620448725 (T) = (0 -1689.0621113637358) / Math.Sqrt((3392.355092158257 / (26)) + (2629.913638880053 / (30))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (30) - 2, .025) and -0.09536784170953873 = (1542.004472878827 - 1689.0621113637358) / 1542.004472878827 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.78 μs 2.29 μs 1.29 0.15 True
1.89 μs 2.18 μs 1.15 0.20 False
1.40 μs 1.85 μs 1.32 0.17 True
1.62 μs 2.22 μs 1.37 0.20 True
2.98 μs 3.76 μs 1.26 0.14 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.287924208277271 > 1.8763702753808094. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -28.9750658186966 (T) = (0 -2336.88875780044) / Math.Sqrt((4773.1329390818855 / (28)) + (3377.7359851861706 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2658729023762727 = (1846.0690274779377 - 2336.88875780044) / 1846.0690274779377 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1783193084110746 > 2.060762295138124. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -17.25383099001487 (T) = (0 -2191.463431590378) / Math.Sqrt((2845.828812678904 / (29)) + (1681.8209448243101 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.10910966278507883 = (1975.8762412072103 - 2191.463431590378) / 1975.8762412072103 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8539979235614636 > 1.459555953735041. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.45222149273408 (T) = (0 -1908.6079282229691) / Math.Sqrt((2412.03913282662 / (30)) + (2785.9504939337526 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.3399652078116916 = (1424.3712576238697 - 1908.6079282229691) / 1424.3712576238697 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.221675224988131 > 1.7416493493901464. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -33.91252030188959 (T) = (0 -2200.0809983304694) / Math.Sqrt((2885.710691811177 / (27)) + (3314.573528701398 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.2974188165048977 = (1695.736928077892 - 2200.0809983304694) / 1695.736928077892 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.757506820101351 > 3.0149040327874346. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.05343142207463 (T) = (0 -3879.8879062012156) / Math.Sqrt((11960.458025959198 / (29)) + (10120.777676553103 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.32672797758989824 = (2924.4034736113167 - 3879.8879062012156) / 2924.4034736113167 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.22 μs 10.69 μs 1.30 0.10 True
10.17 μs 13.45 μs 1.32 0.10 True
43.20 μs 56.94 μs 1.32 0.07 True
12.98 μs 13.80 μs 1.06 0.24 False
657.44 μs 895.83 μs 1.36 0.10 True
28.35 μs 35.99 μs 1.27 0.06 True
262.97 μs 355.04 μs 1.35 0.08 True
9.82 ms 13.37 ms 1.36 0.09 True
4.40 ms 6.06 ms 1.38 0.09 True
294.71 μs 400.29 μs 1.36 0.14 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.690925577875282 > 8.480062786408848. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -42.534491910451806 (T) = (0 -10711.443678136706) / Math.Sqrt((72261.0838490477 / (29)) + (24142.830052689333 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.30049889842984856 = (8236.411188866918 - 10711.443678136706) / 8236.411188866918 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.449708293719388 > 10.887582645736174. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -48.06451029541925 (T) = (0 -13535.195106713903) / Math.Sqrt((70023.56561956351 / (27)) + (46254.00176724127 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.29839587009922863 = (10424.551878526454 - 13535.195106713903) / 10424.551878526454 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 56.944937557656836 > 45.5403191532899. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -53.98936856873076 (T) = (0 -57518.36654427525) / Math.Sqrt((762472.2132445956 / (29)) + (1131820.4249096413 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.31561253514353454 = (43719.83772410617 - 57518.36654427525) / 43719.83772410617 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 1000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.798457530234316 > 13.654085587305724. IsChangePoint: Marked as a change because one of 9/25/2023 4:14:13 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -2.873068938276947 (T) = (0 -14302.0677051009) / Math.Sqrt((1273505.2485319264 / (23)) + (1028965.987525973 / (35))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (35) - 2, .025) and -0.06212112879216883 = (13465.571221019807 - 14302.0677051009) / 13465.571221019807 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 895.8330969907407 > 688.9864173177084. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.11858417692103 (T) = (0 -882684.7769728175) / Math.Sqrt((278424162.89326394 / (29)) + (614696976.5447178 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.3261589071898973 = (665595.0295151339 - 882684.7769728175) / 665595.0295151339 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.988341584564864 > 28.59053535513008. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -41.32156270672474 (T) = (0 -35754.90005967103) / Math.Sqrt((436902.2714770784 / (29)) + (615892.9213769807 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.2861518023385909 = (27799.906663162485 - 35754.90005967103) / 27799.906663162485 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 355.0431600743007 > 285.7486636243761. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -50.195481953467336 (T) = (0 -357742.31239584263) / Math.Sqrt((43948003.94268517 / (29)) + (35603556.17332682 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.30274691252493746 = (274606.1487127068 - 357742.31239584263) / 274606.1487127068 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 1000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.371416316742081 > 10.283733072395835. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -62.73675260637313 (T) = (0 -13775498.067980744) / Math.Sqrt((32031610843.35047 / (28)) + (79716571244.8483 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.39694066466657546 = (9861190.540450554 - 13775498.067980744) / 9861190.540450554 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 1000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.0623463414634156 > 4.582256850585938. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -64.55800434156633 (T) = (0 -6128319.521159499) / Math.Sqrt((10839589632.988588 / (30)) + (10456422442.588835 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.3947718324308661 = (4393779.239489523 - 6128319.521159499) / 4393779.239489523 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 1000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 400.29098828895474 > 304.14212467881947. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -53.167608478571495 (T) = (0 -409036.4542828078) / Math.Sqrt((45160116.52007686 / (28)) + (86628672.01221846 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.38654576669243124 = (295003.93287309486 - 409036.4542828078) / 295003.93287309486 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.39 ms 2.82 ms 1.18 0.04 True
1.16 ms 1.41 ms 1.22 0.05 True
18.68 ms 20.70 ms 1.11 0.01 True
1.09 ms 1.30 ms 1.20 0.12 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.8224598062499995 > 2.5611319520312494. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.069796583499986 (T) = (0 -2814877.4511323622) / Math.Sqrt((6182907361.77974 / (28)) + (1836017372.5552413 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.1443883330403611 = (2459722.254991816 - 2814877.4511323622) / 2459722.254991816 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4123089249999996 > 1.1981045618990387. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.31769401851593 (T) = (0 -1427085.942955905) / Math.Sqrt((823386069.7559215 / (28)) + (554130782.5507469 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.21544697257666315 = (1174124.3963367501 - 1427085.942955905) / 1174124.3963367501 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.701081243750004 > 19.60277432180944. IsChangePoint: Marked as a change because one of 9/26/2023 10:37:55 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -22.705487381495594 (T) = (0 -20443479.544167735) / Math.Sqrt((98817593031.28177 / (28)) + (45329225732.76286 / (30))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (30) - 2, .025) and -0.0855988969589152 = (18831522.03031524 - 20443479.544167735) / 18831522.03031524 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3010598909970237 > 1.131477463. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -29.613466832656798 (T) = (0 -1293629.487407749) / Math.Sqrt((815078747.0932627 / (29)) + (419753528.329768 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.17560922157394765 = (1100390.7282011549 - 1293629.487407749) / 1100390.7282011549 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.13 μs 7.43 μs 1.21 0.13 True
3.47 μs 4.28 μs 1.23 0.03 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<Location>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.4311192014396354 > 6.51207430038178. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -22.302519934954102 (T) = (0 -7548.3616646155115) / Math.Sqrt((54658.42380815201 / (28)) + (23705.5343011347 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.18411906803026826 = (6374.664396859929 - 7548.3616646155115) / 6374.664396859929 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<Location>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.280616856912255 > 3.703750184954831. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.26285727211172 (T) = (0 -4315.389785974526) / Math.Sqrt((6268.8943030875025 / (28)) + (5749.383369686918 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.21567738418352606 = (3549.782073862327 - 4315.389785974526) / 3549.782073862327 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.IO.Tests.Perf_RandomAccess

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
187.18 μs 211.88 μs 1.13 0.09 True
9.71 ms 10.61 ms 1.09 0.02 True
189.94 μs 217.18 μs 1.14 0.06 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_RandomAccess*'
### Payloads [Baseline]() [Compare]() ### System.IO.Tests.Perf_RandomAccess.ReadScatter(fileSize: 1048576, buffersSize: 16384, options: None) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 211.8771412671233 > 198.95751080341012. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.11365097422057 (T) = (0 -220354.50484672392) / Math.Sqrt((15348537.980519965 / (29)) + (33226068.85230147 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.14156195034042796 = (193028.95018620012 - 220354.50484672392) / 193028.95018620012 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.Perf_RandomAccess.ReadScatter(fileSize: 104857600, buffersSize: 65536, options: Asynchronous) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.61054333 > 10.487313422386363. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -17.44218621265172 (T) = (0 -10611254.73511747) / Math.Sqrt((18527862189.688725 / (27)) + (30750557135.51925 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.07376271311354572 = (9882308.824403532 - 10611254.73511747) / 9882308.824403532 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.Perf_RandomAccess.ReadScatter(fileSize: 1048576, buffersSize: 16384, options: Asynchronous) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 217.18053022300467 > 199.42652349542672. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.690975823133098 (T) = (0 -220272.99800824895) / Math.Sqrt((28122578.552952625 / (27)) + (19513271.108970836 / (28))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (28) - 2, .025) and -0.13361539545318307 = (194310.16806206206 - 220272.99800824895) / 194310.16806206206 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Logging.LoggingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
365.61 ns 418.42 ns 1.14 0.18 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.LoggingOverhead*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_DefineMessage #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 418.41774666487595 > 384.51724987555184. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.193456413363018 (T) = (0 -416.3431229691905) / Math.Sqrt((142.95344558734627 / (27)) + (132.6679535600575 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.10132965992554105 = (378.0367841880684 - 416.3431229691905) / 378.0367841880684 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.59 μs 8.72 μs 1.15 0.13 True
1.70 μs 1.87 μs 1.11 0.18 False
2.75 μs 3.04 μs 1.11 0.16 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.724992199599898 > 8.037944460939569. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -26.725393059505812 (T) = (0 -8959.234769629164) / Math.Sqrt((24405.393113352282 / (28)) + (44797.03698070614 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.17182926773471707 = (7645.512035169092 - 8959.234769629164) / 7645.512035169092 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8743001649006181 > 1.7619307316181658. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 4:14:13 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -13.545953514607401 (T) = (0 -1954.2831193898737) / Math.Sqrt((4073.452576114255 / (22)) + (5735.691235040319 / (35))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (35) - 2, .025) and -0.1487596047333134 = (1701.2115601362602 - 1954.2831193898737) / 1701.2115601362602 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.037328064884679 > 2.876063059060202. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 11:21:57 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.211862238793003 (T) = (0 -3040.5629086432646) / Math.Sqrt((7391.716978479693 / (21)) + (6150.352095425551 / (36))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (36) - 2, .025) and -0.09207837952349265 = (2784.1984290266378 - 3040.5629086432646) / 2784.1984290266378 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.07 μs 5.04 μs 1.24 0.28 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;XmlElement&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.044195704439379 > 4.255713471451071. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -20.915125036746662 (T) = (0 -4975.1253236656) / Math.Sqrt((30953.420497067378 / (29)) + (20740.467046059635 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.21579259157202776 = (4092.0839279278143 - 4975.1253236656) / 4092.0839279278143 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Linq.Tests.Perf_OrderBy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.41 ms 1.96 ms 1.40 0.07 True
991.64 μs 1.47 ms 1.48 0.05 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_OrderBy*'
### Payloads [Baseline]() [Compare]() ### System.Linq.Tests.Perf_OrderBy.OrderByValueType(NumberOfPeople: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.96234710546875 > 1.4611091434495191. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -63.214646247207426 (T) = (0 -1967930.498934798) / Math.Sqrt((964051879.4839298 / (28)) + (1278865257.1703224 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.39793302007271875 = (1407743.0539786725 - 1967930.498934798) / 1407743.0539786725 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_OrderBy.OrderByCustomComparer(NumberOfPeople: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4697357901785713 > 1042.3861491346154. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -79.1249154456656 (T) = (0 -1468142.2715942238) / Math.Sqrt((505847010.27139777 / (29)) + (407134965.31519604 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.44373208867122016 = (1016907.6957660962 - 1468142.2715942238) / 1016907.6957660962 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.55 μs 2.05 μs 1.32 0.09 True
2.68 μs 3.20 μs 1.19 0.15 True
3.38 μs 4.04 μs 1.20 0.10 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.050856220928644 > 1.6011460438498673. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -28.546629613343196 (T) = (0 -1895.355272003711) / Math.Sqrt((1651.5856242390848 / (29)) + (2856.375138746805 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.23119741389780507 = (1539.440588981804 - 1895.355272003711) / 1539.440588981804 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2015962697479585 > 2.7837802459549965. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -23.84500436559957 (T) = (0 -3184.315999123761) / Math.Sqrt((7046.07237147937 / (29)) + (5784.08204100313 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.18695262899549012 = (2682.765867260116 - 3184.315999123761) / 2682.765867260116 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.03961082964923 > 3.625590048920463. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.781425853742316 (T) = (0 -4063.652983228653) / Math.Sqrt((12055.572551455101 / (28)) + (6701.570404480387 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.14313692482063753 = (3554.826106125699 - 4063.652983228653) / 3554.826106125699 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.65 μs 6.10 μs 1.31 0.07 True
5.04 μs 6.01 μs 1.19 0.06 True
6.39 μs 7.86 μs 1.23 0.06 True
4.39 μs 5.58 μs 1.27 0.04 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.100121958612039 > 4.893819430380011. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -38.35508699338446 (T) = (0 -5950.972467653864) / Math.Sqrt((13201.722346016113 / (29)) + (16472.7255406691 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2597169762002305 = (4724.055148962258 - 5950.972467653864) / 4724.055148962258 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.009344737095532 > 5.176378505002741. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -35.43045893605337 (T) = (0 -6186.240642991922) / Math.Sqrt((18890.917053561723 / (28)) + (13957.252316359087 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2418115896525153 = (4981.62579133519 - 6186.240642991922) / 4981.62579133519 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.860685046329615 > 6.572145865703433. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.78155390990354 (T) = (0 -7748.9308047567165) / Math.Sqrt((26532.68961184895 / (29)) + (14508.493033815308 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.22462119877017828 = (6327.614459506788 - 7748.9308047567165) / 6327.614459506788 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.581675287510428 > 4.6544278172497355. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.62629689527623 (T) = (0 -5656.305405747148) / Math.Sqrt((13462.164919363602 / (29)) + (13855.261090580922 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.256540552221084 = (4501.4905374513855 - 5656.305405747148) / 4501.4905374513855 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.23 ms 1.40 ms 1.14 0.10 True
1.60 ms 2.08 ms 1.30 0.12 True
2.37 ms 3.01 ms 1.27 0.06 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4033859261363637 > 1.2902492575490017. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.915855137377385 (T) = (0 -1409518.4819108818) / Math.Sqrt((1388835271.0146396 / (29)) + (478181373.1852738 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.12786789451374891 = (1249719.4828996877 - 1409518.4819108818) / 1249719.4828996877 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.0750756291316526 > 1.6742073386522294. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -30.116641632981885 (T) = (0 -2097164.612735372) / Math.Sqrt((4104011697.9007196 / (28)) + (2616868351.7342153 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2836807470751422 = (1633711.978242057 - 2097164.612735372) / 1633711.978242057 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.008985366609294 > 2.550751907860311. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -39.18364126748518 (T) = (0 -2961121.5366121666) / Math.Sqrt((3169394516.55685 / (29)) + (1722988623.4877074 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.20911873131643904 = (2448991.5340143796 - 2961121.5366121666) / 2448991.5340143796 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
606.79 μs 777.53 μs 1.28 0.08 True
1.88 ms 2.62 ms 1.39 0.10 True
590.71 μs 774.41 μs 1.31 0.16 True
4.75 ms 5.81 ms 1.22 0.06 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 777.5280920335431 > 625.1656321443722. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -42.256580087150475 (T) = (0 -755742.251285131) / Math.Sqrt((241638011.02397126 / (29)) + (117699923.21311578 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.24505369739270216 = (606995.7085929302 - 755742.251285131) / 606995.7085929302 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.619229785815603 > 1.9809000273449577. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -63.115216132864006 (T) = (0 -2619956.235010918) / Math.Sqrt((1289037670.3591068 / (29)) + (2309991931.6544843 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.3786930921574461 = (1900318.6785472922 - 2619956.235010918) / 1900318.6785472922 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 774.406454375 > 623.8734671967455. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -43.37393358859727 (T) = (0 -763682.7785422145) / Math.Sqrt((235274191.4310977 / (29)) + (154514682.72717765 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2629840424538204 = (604665.4216299314 - 763682.7785422145) / 604665.4216299314 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.811008503401361 > 5.004172373543551. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.080652621203345 (T) = (0 -5754644.511129623) / Math.Sqrt((35186140958.958534 / (28)) + (12188032014.62691 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.17648545298303625 = (4891386.031623631 - 5754644.511129623) / 4891386.031623631 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.52 ms 1.87 ms 1.23 0.08 True
1.51 ms 1.83 ms 1.21 0.05 True
1.56 ms 1.87 ms 1.20 0.04 True
1.53 ms 1.83 ms 1.20 0.07 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.869685648611111 > 1.5768190911363635. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.73155272254143 (T) = (0 -1813665.3827649911) / Math.Sqrt((891364123.2289145 / (29)) + (957734382.0573777 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.19291852418487485 = (1520359.8116680053 - 1813665.3827649911) / 1520359.8116680053 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8269160052083333 > 1.615958484730903. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.367179647578034 (T) = (0 -1823501.4595902269) / Math.Sqrt((916071435.2014513 / (29)) + (881081790.5738933 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.1862383021891622 = (1537213.4386699682 - 1823501.4595902269) / 1537213.4386699682 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8710223141741071 > 1.6411741306826924. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.39485533031273 (T) = (0 -1852261.7367596582) / Math.Sqrt((767593080.4399408 / (29)) + (1016361030.627863 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.18218664155361253 = (1566809.9026440044 - 1852261.7367596582) / 1566809.9026440044 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8306765952380954 > 1.6081872843750002. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.652533964800654 (T) = (0 -1861002.9637124708) / Math.Sqrt((2174947996.5411477 / (28)) + (919347111.5537716 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.18399368479566833 = (1571801.4273307882 - 1861002.9637124708) / 1571801.4273307882 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Logging.Formatting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.70 μs 1.99 μs 1.17 0.11 True
386.16 ns 436.92 ns 1.13 0.17 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.Formatting*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.Logging.Formatting.TwoArguments #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9863994418871027 > 1.7741257393117311. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -24.64178498882849 (T) = (0 -1996.7873926077634) / Math.Sqrt((1110.6684834773312 / (28)) + (2777.9189192267236 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.16771774157216482 = (1709.9914829754791 - 1996.7873926077634) / 1709.9914829754791 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.Formatting.NoArguments #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 436.9166513427883 > 410.3675294766831. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -11.79601147425612 (T) = (0 -427.8075459827404) / Math.Sqrt((129.46109798981195 / (29)) + (94.2826600378409 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.08294073939166068 = (395.04243438386135 - 427.8075459827404) / 395.04243438386135 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.28 μs 11.76 μs 1.14 0.08 True
4.30 μs 5.29 μs 1.23 0.20 True
17.22 μs 19.97 μs 1.16 0.09 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.759386311083192 > 10.931659942210723. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -19.80016166997639 (T) = (0 -11783.734606426338) / Math.Sqrt((74730.76921385569 / (29)) + (35572.818681042045 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.11560958468348562 = (10562.59713811042 - 11783.734606426338) / 10562.59713811042 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.290001169590643 > 4.372809492948546. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -28.373627820724973 (T) = (0 -5195.422508661361) / Math.Sqrt((17139.717006355855 / (28)) + (11577.658417600705 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.21018349711699336 = (4293.086561697757 - 5195.422508661361) / 4293.086561697757 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.97076947810374 > 17.939814968177394. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -17.121650066090243 (T) = (0 -19774.42064146867) / Math.Sqrt((430103.312955496 / (29)) + (127387.8209352201 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.13773620081654947 = (17380.49701440161 - 19774.42064146867) / 17380.49701440161 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Threading.Channels.Tests.BoundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
562.01 ns 805.88 ns 1.43 0.24 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### System.Threading.Channels.Tests.BoundedChannelPerfTests.TryWriteThenTryRead #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 805.8794818312487 > 594.4365749893622. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -49.90124985232369 (T) = (0 -834.1230539440036) / Math.Sqrt((475.6595958305464 / (29)) + (280.0953465784793 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.43968299990047904 = (579.3796648301494 - 834.1230539440036) / 579.3796648301494 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.00 secs 10.05 secs 1.44 0.08 True
2.08 secs 2.44 secs 1.17 0.49 False
7.09 secs 9.87 secs 1.39 0.03 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'
### Payloads [Baseline]() [Compare]() ### LinqBenchmarks.Order00LinqMethodX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.051284333066668 > 7.493023625623847. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -67.9109584253157 (T) = (0 -10112266276.932638) / Math.Sqrt((15176740323312272 / (29)) + (37853579728914984 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.4028790932567031 = (7208223663.421766 - 10112266276.932638) / 7208223663.421766 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### LinqBenchmarks.Where00LinqQueryX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.439070615230769 > 2.1757029999750004. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/25/2023 9:33:39 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -4.325216109230102 (T) = (0 -2313007934.423949) / Math.Sqrt((24640651684629796 / (20)) + (13478199199645220 / (37))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (37) - 2, .025) and -0.08074432946041021 = (2140199001.1631875 - 2313007934.423949) / 2140199001.1631875 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### LinqBenchmarks.Order00LinqQueryX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.865209999933333 > 7.438695284685. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -84.12138410295547 (T) = (0 -10065776258.11051) / Math.Sqrt((13390587724984128 / (27)) + (19936574514065204 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.4034956878424313 = (7171932443.614734 - 10065776258.11051) / 7171932443.614734 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.02 μs 10.29 μs 1.14 0.12 False
2.50 μs 2.88 μs 1.15 0.19 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;ClassImplementingIXmlSerialiable&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.285542926548091 > 9.418179576390953. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -14.236519580432372 (T) = (0 -10248.910211460416) / Math.Sqrt((125016.10705576961 / (27)) + (41045.38228393317 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.1210829602170716 = (9141.973052088806 - 10248.910211460416) / 9141.973052088806 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.8817769286246047 > 2.6618432172958983. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/24/2023 11:15:45 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -12.271791859308763 (T) = (0 -2775.5273912602615) / Math.Sqrt((3280.210287994013 / (20)) + (5148.25580906784 / (39))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (39) - 2, .025) and -0.08233460115905455 = (2564.3894118214407 - 2775.5273912602615) / 2564.3894118214407 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.55 μs 12.00 μs 1.14 0.08 False
38.85 μs 43.78 μs 1.13 0.08 False
4.39 μs 5.08 μs 1.16 0.06 True
5.29 μs 6.46 μs 1.22 0.14 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromStream<Location>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.995327700972863 > 10.820564541974205. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -15.63838779365921 (T) = (0 -11838.583276963747) / Math.Sqrt((154545.8634353592 / (27)) + (44246.39227762312 / (28))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (28) - 2, .025) and -0.12726305200040955 = (10502.059174169975 - 11838.583276963747) / 10502.059174169975 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<Location>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 43.78382980147737 > 39.75050373278925. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -18.327512381030612 (T) = (0 -42011.41537684291) / Math.Sqrt((611490.4953017366 / (28)) + (489097.64231748745 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.09388304546045037 = (38405.76517863385 - 42011.41537684291) / 38405.76517863385 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<Location>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.082377658093054 > 4.575909202885336. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -18.492955218229845 (T) = (0 -5039.237167404759) / Math.Sqrt((18756.466980057427 / (27)) + (7265.202224130712 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.12717330211982486 = (4470.685348852469 - 5039.237167404759) / 4470.685348852469 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<Location>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.460142591424969 > 5.555877534701766. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.74036234135758 (T) = (0 -6597.482151331767) / Math.Sqrt((15399.495642154156 / (29)) + (21737.56358472123 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.23218673540906368 = (5354.287594356811 - 6597.482151331767) / 5354.287594356811 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.14 μs 6.37 μs 1.24 0.09 True
4.40 μs 4.99 μs 1.13 0.08 True
9.50 μs 11.50 μs 1.21 0.10 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.372426676829267 > 5.427585529533417. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.41710201087725 (T) = (0 -6577.775858643262) / Math.Sqrt((24663.450046990227 / (27)) + (14283.772725239423 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.24408333850177893 = (5287.246967365327 - 6577.775858643262) / 5287.246967365327 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.987933255315774 > 4.603951901359281. IsChangePoint: Marked as a change because one of 9/26/2023 10:37:55 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -18.78973167734388 (T) = (0 -5025.768191375226) / Math.Sqrt((19739.736280803398 / (27)) + (6314.289972958582 / (30))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (30) - 2, .025) and -0.12958850035220168 = (4449.202687357572 - 5025.768191375226) / 4449.202687357572 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<Location>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.495601372111347 > 9.740707319985228. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -17.01658121801877 (T) = (0 -11124.568009913524) / Math.Sqrt((186883.25514030264 / (28)) + (64753.02531173065 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.1687223833818148 = (9518.57187651654 - 11124.568009913524) / 9518.57187651654 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
105.76 μs 140.99 μs 1.33 0.10 True
120.64 μs 155.98 μs 1.29 0.09 True
106.29 μs 140.91 μs 1.33 0.07 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;TreeRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 140.98717016570146 > 110.89271544430272. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.64352273139133 (T) = (0 -135706.85433155924) / Math.Sqrt((6548640.6002994785 / (28)) + (9517916.289067231 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2625695878862691 = (107484.65322909676 - 135706.85433155924) / 107484.65322909676 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 155.98468767857142 > 126.68940493853805. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -38.99432445423292 (T) = (0 -151799.7212869743) / Math.Sqrt((8330505.776098176 / (28)) + (6749902.703189514 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.23010268430530662 = (123404.10538385445 - 151799.7212869743) / 123404.10538385445 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 140.90723465909093 > 112.55879072956985. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -36.030712892949985 (T) = (0 -136322.25366098539) / Math.Sqrt((8637490.692718383 / (29)) + (7859612.277888655 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.24898140552923795 = (109146.7439446953 - 136322.25366098539) / 109146.7439446953 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.IO.Tests.Perf_FileStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.34 μs 10.41 μs 1.11 0.22 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileStream*'
### Payloads [Baseline]() [Compare]() ### System.IO.Tests.Perf_FileStream.Read(fileSize: 1024, userBufferSize: 1024, options: Asynchronous) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.414755414188146 > 9.724077361129304. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -6.93496850454458 (T) = (0 -10214.7931718514) / Math.Sqrt((126509.09732664275 / (27)) + (109684.64621157192 / (29))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (29) - 2, .025) and -0.06663702789543745 = (9576.634698315345 - 10214.7931718514) / 9576.634698315345 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
124.40 μs 150.66 μs 1.21 0.09 True
225.08 μs 269.10 μs 1.20 0.09 True
156.49 μs 200.92 μs 1.28 0.07 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 150.65592580436387 > 130.30693820684522. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -31.09145045275355 (T) = (0 -149462.2262133392) / Math.Sqrt((5246854.924262965 / (29)) + (10915578.619700659 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.18925075708706984 = (125677.6380613197 - 149462.2262133392) / 125677.6380613197 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 269.09985625 > 237.43628207917948. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -26.854561082464567 (T) = (0 -264287.0764502919) / Math.Sqrt((35571728.761264615 / (29)) + (14406182.421333125 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.15392524683062692 = (229033.1000003538 - 264287.0764502919) / 229033.1000003538 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 200.9226945201818 > 163.54096306928056. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -29.021607906799485 (T) = (0 -195738.18716936614) / Math.Sqrt((28379123.952970233 / (28)) + (15005142.455969425 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.22467105225838294 = (159829.19397695456 - 195738.18716936614) / 159829.19397695456 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
138.44 μs 202.85 μs 1.47 0.18 True
794.90 ns 1.12 μs 1.41 0.11 True
137.53 μs 202.29 μs 1.47 0.12 True
643.04 ns 882.23 ns 1.37 0.16 True
138.58 μs 206.18 μs 1.49 0.19 True
137.83 μs 205.38 μs 1.49 0.10 True
774.68 ns 1.09 μs 1.41 0.14 True
657.99 ns 897.16 ns 1.36 0.17 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'
### Payloads [Baseline]() [Compare]() ### System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsLarge #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 202.85428981755425 > 143.53197577991094. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -48.868717885122265 (T) = (0 -199553.41501744575) / Math.Sqrt((18407884.98087066 / (28)) + (25028309.915487405 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.4325986982864217 = (139294.7063655287 - 199553.41501744575) / 139294.7063655287 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1240001610941697 > 826.616008904421. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -45.21919839059928 (T) = (0 -1119.3184013733646) / Math.Sqrt((915.7584709127398 / (29)) + (550.0775559100131 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.40674107417910793 = (795.6818933623116 - 1119.3184013733646) / 795.6818933623116 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 202.28542451298702 > 143.15640580527747. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -65.08972675461038 (T) = (0 -202278.9521696054) / Math.Sqrt((9566885.644017044 / (29)) + (17433569.808881346 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.45030631460755033 = (139473.26170495345 - 202278.9521696054) / 139473.26170495345 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsSmall #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 882.2283732253961 > 676.501686347176. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -46.83306781248799 (T) = (0 -897.0666604793689) / Math.Sqrt((332.4868464833175 / (28)) + (443.9446905285889 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.3739898917670019 = (652.8917467694816 - 897.0666604793689) / 652.8917467694816 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityLinksLarge #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 206.17634679487176 > 145.43592259615383. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -62.62732407536061 (T) = (0 -206739.9212784496) / Math.Sqrt((9714901.142728295 / (29)) + (22032946.32184904 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.4640324555145983 = (141212.6626699555 - 206739.9212784496) / 141212.6626699555 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityEventsLarge #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 205.38474171305666 > 147.26542641225964. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -47.245620678535936 (T) = (0 -201656.75569816848) / Math.Sqrt((18356068.79853954 / (29)) + (27294277.900304813 / (27))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (27) - 2, .025) and -0.42936307082027986 = (141081.54870857418 - 201656.75569816848) / 141081.54870857418 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.08850826866616 > 811.8212844679265. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -59.694739654475484 (T) = (0 -1097.079535503671) / Math.Sqrt((464.5616667849784 / (28)) + (301.58094717712356 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.394091865210074 = (786.9492412096913 - 1097.079535503671) / 786.9492412096913 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsSmall #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 897.1642369901989 > 676.5737638326689. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -55.075489471022244 (T) = (0 -903.3388370763748) / Math.Sqrt((297.66057810111226 / (29)) + (293.21568489242753 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.37970123445392867 = (654.7351082380577 - 903.3388370763748) / 654.7351082380577 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.60 ms 2.13 ms 1.33 0.05 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CreateAddAndRemove<Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1297872608173076 > 1.666551560859375. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -45.072934828824465 (T) = (0 -2125070.841664073) / Math.Sqrt((1233761438.906769 / (28)) + (2463774840.5443497 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.3173834343332974 = (1613099.7143891754 - 2125070.841664073) / 1613099.7143891754 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.49 μs 3.47 μs 1.39 0.05 True
574.53 ns 741.21 ns 1.29 0.12 True
117.24 ns 133.10 ns 1.14 0.34 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.GetService.Transient #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.4653609151272344 > 2.544069610075332. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -54.355238575808876 (T) = (0 -3508.901680101821) / Math.Sqrt((4795.327891162161 / (29)) + (5952.09310210028 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.43091304793055113 = (2452.2116736419084 - 3508.901680101821) / 2452.2116736419084 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.GetService.Scoped #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 741.2104436033005 > 594.2235551405238. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.72671310841513 (T) = (0 -728.2925944075098) / Math.Sqrt((275.8172064520307 / (28)) + (242.94226794412003 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2556125390521738 = (580.0297239443604 - 728.2925944075098) / 580.0297239443604 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.GetService.Singleton #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 133.10078562252318 > 132.20319877569426. IsChangePoint: Marked as a change because one of 8/14/2023 10:44:26 PM, 9/25/2023 9:33:39 AM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -8.47851509203013 (T) = (0 -137.8556338991988) / Math.Sqrt((23.198012146080295 / (21)) + (24.40841942549974 / (37))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (37) - 2, .025) and -0.08896118800785605 = (126.59370730318837 - 137.8556338991988) / 126.59370730318837 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.91 μs 2.42 μs 1.27 0.17 True
4.32 μs 5.01 μs 1.16 0.17 False
2.23 μs 2.91 μs 1.31 0.13 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4193348311850498 > 2.067776694611069. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -26.003852653506158 (T) = (0 -2416.9704772198756) / Math.Sqrt((5366.9032239330945 / (28)) + (1687.9017317044324 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2049227113357225 = (2005.913287617039 - 2416.9704772198756) / 2005.913287617039 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.008469333333332 > 4.394923708883212. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -16.272916908653542 (T) = (0 -4910.041554143345) / Math.Sqrt((19762.766617444737 / (28)) + (21482.431949734826 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.14423387080695615 = (4291.117121607842 - 4910.041554143345) / 4291.117121607842 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.910114235556227 > 2.3598668458753034. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.083126485776987 (T) = (0 -2946.118711565625) / Math.Sqrt((10562.540009220458 / (28)) + (5383.256021468157 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2789312228444145 = (2303.578690504789 - 2946.118711565625) / 2303.578690504789 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Exceptions.Handling

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.03 μs 9.81 μs 1.22 0.15 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Exceptions.Handling*'
### Payloads [Baseline]() [Compare]() ### Exceptions.Handling.ThrowAndCatchFinally(kind: Hardware) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.809765279649595 > 8.440742251652043. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -34.605150621873044 (T) = (0 -9666.22919323114) / Math.Sqrt((28599.125544212817 / (29)) + (30809.910992954807 / (28))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (28) - 2, .025) and -0.19549938891608737 = (8085.515796034938 - 9666.22919323114) / 8085.515796034938 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Threading.Channels.Tests.UnboundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
355.59 ns 420.86 ns 1.18 0.25 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.UnboundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### System.Threading.Channels.Tests.UnboundedChannelPerfTests.TryWriteThenTryRead #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 420.8593033877302 > 366.4159148029478. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -21.38393297419474 (T) = (0 -434.1124311281183) / Math.Sqrt((134.9368946132519 / (28)) + (343.2641295591328 / (29))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (29) - 2, .025) and -0.2516173500460922 = (346.8411740314495 - 434.1124311281183) / 346.8411740314495 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.24 ms 2.88 ms 1.28 0.10 True
3.13 ms 3.65 ms 1.16 0.05 True
1.24 ms 1.53 ms 1.23 0.14 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.8753739487394956 > 2.3149728134374996. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -37.472779526183785 (T) = (0 -2852486.655939984) / Math.Sqrt((5919166065.751359 / (29)) + (1563771664.2596004 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.26746372727897755 = (2250546.9738875865 - 2852486.655939984) / 2250546.9738875865 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.649034117270789 > 3.129118362367886. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -32.528435495201535 (T) = (0 -3725420.5230540084) / Math.Sqrt((8116031668.714828 / (29)) + (4117088833.5827365 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.2185193690468936 = (3057333.857538902 - 3725420.5230540084) / 3057333.857538902 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.534032124562937 > 1.3565573624098557. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/2/2023 10:24:51 PM falls between 9/24/2023 12:29:05 AM and 10/2/2023 10:24:51 PM. IsRegressionStdDev: Marked as regression because -27.664498233107178 (T) = (0 -1517732.107502688) / Math.Sqrt((1352571242.7526965 / (29)) + (397384817.4338698 / (29))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (29) - 2, .025) and -0.16494876766478775 = (1302831.6348581375 - 1517732.107502688) / 1302831.6348581375 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 1 year ago

looks like diplicate of https://github.com/dotnet/perf-autofiling-issues/issues/22385