search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 454 Regressions on 9/26/2023 5:24:05 PM #22385

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.34 μs 5.10 μs 1.18 0.20 False
241.36 μs 258.09 μs 1.07 0.04 False
8.37 μs 9.40 μs 1.12 0.14 False
8.57 μs 9.84 μs 1.15 0.11 False
9.62 μs 10.87 μs 1.13 0.11 False
5.49 μs 6.69 μs 1.22 0.13 False
7.24 μs 8.05 μs 1.11 0.13 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedJsonSerializerContext #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.100523353942944 > 4.4702239647536155. IsChangePoint: Marked as a change because one of 8/21/2023 10:18:47 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.601865147504437 (T) = (0 -5004.356264736374) / Math.Sqrt((13090.965074921933 / (51)) + (11369.307920302243 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17828377012597793 = (4247.157086956503 - 5004.356264736374) / 4247.157086956503 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomConverter #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 258.0935465286832 > 250.6117123201885. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 9/25/2023 10:28:34 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.125609786452841 (T) = (0 -256062.12686409106) / Math.Sqrt((15271883.973650124 / (46)) + (28406660.998105086 / (8))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (8) - 2, .025) and -0.07552883728335538 = (238080.20574406016 - 256062.12686409106) / 238080.20574406016 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewDefaultOptions #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.39675822790876 > 8.753806682573117. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.345817462796044 (T) = (0 -9644.008927213637) / Math.Sqrt((55536.523787042395 / (50)) + (27969.413307408977 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.15460613477651597 = (8352.639603010874 - 9644.008927213637) / 8352.639603010874 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomizedOptions #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.844645757831696 > 9.011393570361816. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.574256900191573 (T) = (0 -9947.613130381598) / Math.Sqrt((38030.29283586491 / (50)) + (43208.476843960874 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.14300546605650022 = (8703.03198522926 - 9947.613130381598) / 8703.03198522926 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCachedCustomConverter #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.874584178946192 > 9.858509310326363. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.31964104446051 (T) = (0 -11006.718833541054) / Math.Sqrt((71108.33539221596 / (50)) + (18641.102206751137 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.15942348460916644 = (9493.26883546036 - 11006.718833541054) / 9493.26883546036 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedDefaultOptions #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.690192466813368 > 5.791846481442025. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.3854792821721 (T) = (0 -6860.241281141385) / Math.Sqrt((22187.885913897528 / (51)) + (27715.5753048464 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.25780288484532726 = (5454.146562865447 - 6860.241281141385) / 5454.146562865447 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewJsonSerializerContext #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.048988382841117 > 7.368679430049158. IsChangePoint: Marked as a change because one of 8/22/2023 10:50:09 AM, 9/25/2023 2:54:53 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.786029631897627 (T) = (0 -7735.024634582908) / Math.Sqrt((27701.675285527694 / (42)) + (37432.59730869336 / (12))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (12) - 2, .025) and -0.08433201875563279 = (7133.446675732711 - 7735.024634582908) / 7133.446675732711 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
26.27 μs 43.76 μs 1.67 0.25 True
2.97 μs 3.46 μs 1.17 0.23 False
1.06 μs 1.23 μs 1.16 0.27 False
20.73 μs 30.83 μs 1.49 0.14 True
72.75 μs 80.80 μs 1.11 0.28 False
23.66 μs 37.56 μs 1.59 0.24 True
21.92 μs 30.90 μs 1.41 0.11 False
11.92 μs 17.64 μs 1.48 0.29 False
24.25 μs 36.58 μs 1.51 0.15 True
15.59 μs 23.48 μs 1.51 0.34 True
2.43 μs 2.67 μs 1.10 0.29 False
3.24 μs 3.55 μs 1.10 0.48 False
14.11 μs 18.48 μs 1.31 0.22 False
15.38 μs 23.03 μs 1.50 0.26 False
23.05 μs 35.70 μs 1.55 0.19 True
85.64 μs 143.83 μs 1.68 0.15 True
22.90 μs 37.27 μs 1.63 0.24 True
3.27 μs 3.59 μs 1.10 0.46 False
3.93 μs 4.28 μs 1.09 0.37 False
12.73 μs 17.35 μs 1.36 0.40 False
7.90 μs 10.46 μs 1.32 0.34 False
3.16 μs 3.89 μs 1.23 0.39 False
22.42 μs 28.51 μs 1.27 0.27 False
13.91 μs 21.58 μs 1.55 0.26 True
14.97 μs 22.91 μs 1.53 0.30 True
23.84 μs 36.93 μs 1.55 0.20 True
3.18 μs 3.59 μs 1.13 0.33 False
40.93 μs 63.44 μs 1.55 0.12 True
13.05 μs 14.07 μs 1.08 0.36 False
79.69 μs 144.06 μs 1.81 0.09 True
23.49 μs 37.01 μs 1.58 0.24 False
20.74 μs 29.14 μs 1.40 0.14 True
3.37 μs 4.03 μs 1.20 0.49 False
22.63 μs 31.62 μs 1.40 0.23 True
35.31 μs 48.57 μs 1.38 0.14 False
3.22 μs 3.56 μs 1.11 0.38 False
8.18 μs 11.59 μs 1.42 0.21 False
26.90 μs 40.84 μs 1.52 0.18 True
41.61 μs 65.83 μs 1.58 0.10 True
39.52 μs 57.60 μs 1.46 0.15 True
37.10 μs 55.91 μs 1.51 0.13 False
26.60 μs 44.60 μs 1.68 0.34 True
24.89 μs 37.01 μs 1.49 0.22 False
19.35 μs 28.74 μs 1.49 0.18 True
19.35 μs 28.32 μs 1.46 0.22 True
18.67 μs 28.98 μs 1.55 0.20 True
585.97 ns 641.70 ns 1.10 0.17 False
20.05 μs 29.88 μs 1.49 0.31 True
3.28 μs 3.74 μs 1.14 0.37 False
4.45 μs 5.17 μs 1.16 0.44 False
19.55 μs 29.11 μs 1.49 0.28 True
8.69 μs 11.04 μs 1.27 0.27 True
82.89 μs 142.72 μs 1.72 0.14 True
8.27 μs 11.47 μs 1.39 0.41 False
11.97 μs 17.50 μs 1.46 0.32 False
121.26 μs 215.41 μs 1.78 0.09 True
9.75 μs 10.58 μs 1.09 0.26 False
8.07 μs 10.89 μs 1.35 0.31 False
26.52 μs 42.08 μs 1.59 0.19 True
28.24 μs 44.28 μs 1.57 0.34 False
22.72 μs 35.09 μs 1.54 0.23 False
13.01 μs 17.58 μs 1.35 0.21 False
104.08 μs 180.53 μs 1.73 0.15 True
33.94 μs 47.85 μs 1.41 0.10 True
23.39 μs 39.34 μs 1.68 0.20 True
3.19 μs 3.73 μs 1.17 0.40 False
811.43 ns 953.48 ns 1.18 0.14 False
2.18 μs 2.34 μs 1.07 0.28 False
3.89 μs 4.38 μs 1.13 0.36 False
23.33 μs 37.67 μs 1.61 0.22 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### System.Linq.Tests.Perf_Enumerable.Take_All(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 43.76206043412476 > 27.08793556935937. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -74.2542243605858 (T) = (0 -43438.97635877944) / Math.Sqrt((910394.1843902959 / (50)) + (144610.28857807736 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.6626493978853887 = (26126.359780977597 - 43438.97635877944) / 26126.359780977597 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.461064707442777 > 3.0780579850091887. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.202679942726265 (T) = (0 -3356.5226912469116) / Math.Sqrt((6231.4163955119875 / (39)) + (24341.345921929733 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.13566728668106126 = (2955.551093715312 - 3356.5226912469116) / 2955.551093715312 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2303460733303815 > 1.1144408040877207. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.160172623755372 (T) = (0 -1177.8693979253803) / Math.Sqrt((766.4118229108827 / (39)) + (2420.4468697071748 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.12096501756782634 = (1050.7637432620513 - 1177.8693979253803) / 1050.7637432620513 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 30.831156996976937 > 22.058564798910602. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -25.24202569051479 (T) = (0 -31159.692765255146) / Math.Sqrt((351677.563443753 / (49)) + (573747.0952101754 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.45854713686871235 = (21363.514402524183 - 31159.692765255146) / 21363.514402524183 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 80.79634321853743 > 76.51089044567078. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.752219359379486 (T) = (0 -80930.59171546262) / Math.Sqrt((7943207.796992523 / (41)) + (5171008.334697896 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.12229378268501238 = (72111.77052219039 - 80930.59171546262) / 72111.77052219039 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.56334775452044 > 25.361782980570073. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -24.408880131853337 (T) = (0 -36654.06647312567) / Math.Sqrt((1500140.2692960585 / (51)) + (720585.7203407387 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4385306975224717 = (25480.211535460185 - 36654.06647312567) / 25480.211535460185 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 30.901252472167652 > 22.986836955996388. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -25.33851581218601 (T) = (0 -31565.1155848793) / Math.Sqrt((1084384.982221772 / (48)) + (316867.6969051189 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.34432612310566857 = (23480.25158654019 - 31565.1155848793) / 23480.25158654019 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.643105712256492 > 12.515566964143327. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.641973357740639 (T) = (0 -17473.113020120258) / Math.Sqrt((885056.2448162014 / (49)) + (405566.5427062642 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.26662455797646695 = (13795.021508216247 - 17473.113020120258) / 13795.021508216247 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 36.57618685302516 > 24.889329721143895. IsChangePoint: Marked as a change because one of 9/19/2023 12:42:36 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -31.098682301025246 (T) = (0 -36469.286332587595) / Math.Sqrt((1626872.6465525217 / (49)) + (403661.94056934846 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4540990245910753 = (25080.332024047373 - 36469.286332587595) / 25080.332024047373 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Select(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.479846926627797 > 15.611513204295287. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.787243642833777 (T) = (0 -21884.175549899195) / Math.Sqrt((436660.17365650315 / (49)) + (1407247.2297986026 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4781882128926673 = (14804.72876121373 - 21884.175549899195) / 14804.72876121373 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.669029827235772 > 2.654861871828226. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.205413586087894 (T) = (0 -2827.176152422456) / Math.Sqrt((37212.49674999803 / (41)) + (28453.011108724724 / (13))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (13) - 2, .025) and -0.11414785990306257 = (2537.523298450205 - 2827.176152422456) / 2537.523298450205 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.5529979918890633 > 3.3901385546303953. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.344887605863168 (T) = (0 -3717.8102459368297) / Math.Sqrt((27216.972212802757 / (38)) + (21879.711345446973 / (14))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (14) - 2, .025) and -0.13635817723935978 = (3271.6887337131548 - 3717.8102459368297) / 3271.6887337131548 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.476456690081246 > 12.846855992969262. IsChangePoint: Marked as a change because one of 9/19/2023 5:34:53 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -24.018582309514162 (T) = (0 -18400.353233933736) / Math.Sqrt((1156820.249256477 / (49)) + (39413.41690814158 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.31367949351367946 = (14006.729438029499 - 18400.353233933736) / 14006.729438029499 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.026231053841997 > 16.7925682710665. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.359864221862686 (T) = (0 -23053.707234668494) / Math.Sqrt((657840.117307801 / (48)) + (364481.8182317971 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.45777530325249177 = (15814.307721658193 - 23053.707234668494) / 15814.307721658193 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.70240273646071 > 24.575215389808772. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.53207622949979 (T) = (0 -36188.37612303959) / Math.Sqrt((1802479.7646629268 / (48)) + (305569.093576691 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.43564877554324144 = (25206.98428440209 - 36188.37612303959) / 25206.98428440209 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.OrderByThenBy(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 143.83383472222224 > 90.01589252285626. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -38.430749466552875 (T) = (0 -143871.00896208175) / Math.Sqrt((6938528.097746182 / (49)) + (7834521.387566977 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.6316258056974703 = (88176.47309799766 - 143871.00896208175) / 88176.47309799766 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.27142717391304 > 25.679231347071482. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -49.28487276318413 (T) = (0 -37915.778041228354) / Math.Sqrt((641158.7630268905 / (48)) + (259370.44512512864 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.5710771710038521 = (24133.619112421937 - 37915.778041228354) / 24133.619112421937 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.5936665507286008 > 3.405322034573069. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.346782555975985 (T) = (0 -3644.3106022727834) / Math.Sqrt((20396.138594750864 / (40)) + (18871.54445954364 / (13))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (13) - 2, .025) and -0.1281486256321637 = (3230.3461791044383 - 3644.3106022727834) / 3230.3461791044383 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.276155832867698 > 4.0980790694017655. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -4.765731698138736 (T) = (0 -4162.542562256385) / Math.Sqrt((21992.268898310405 / (39)) + (54529.776994683154 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.08277979489133076 = (3844.31126429926 - 4162.542562256385) / 3844.31126429926 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.346282398148144 > 13.367018013506051. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.289483231557856 (T) = (0 -14990.124468688577) / Math.Sqrt((255677.75200406162 / (12)) + (1364015.3061347327 / (42))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (42) - 2, .025) and -0.1893268443194745 = (12603.872972585457 - 14990.124468688577) / 12603.872972585457 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.460952363095236 > 8.429306393703737. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.428271443973369 (T) = (0 -10830.911957334296) / Math.Sqrt((140151.3665712502 / (50)) + (158566.24962209963 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.34305107540407676 = (8064.408089674219 - 10830.911957334296) / 8064.408089674219 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.8880023763987204 > 3.418196828669158. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.601877753375796 (T) = (0 -3727.342526279697) / Math.Sqrt((10716.348588249934 / (40)) + (34890.27135670842 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.15652012497166465 = (3222.8946524999023 - 3727.342526279697) / 3222.8946524999023 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Except(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.511025076034063 > 22.370361168180892. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.28630288424859 (T) = (0 -28124.284095600065) / Math.Sqrt((1642342.9574806513 / (49)) + (919713.6123171696 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.20405835646586862 = (23357.907816154344 - 28124.284095600065) / 23357.907816154344 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.576718644446647 > 15.244814397868563. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.069877936101438 (T) = (0 -21399.747758961523) / Math.Sqrt((403175.5839128292 / (51)) + (197863.3031697682 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4346945136003466 = (14915.891540742805 - 21399.747758961523) / 14915.891540742805 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Where(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.90719487958989 > 15.866240581280893. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.41845579379482 (T) = (0 -21789.313390872343) / Math.Sqrt((404593.0496691293 / (49)) + (690501.0410598568 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4305142233482421 = (15231.80478406749 - 21789.313390872343) / 15231.80478406749 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 36.92983891454966 > 25.410630941389098. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.981951819679242 (T) = (0 -36692.200350321495) / Math.Sqrt((2153402.43908407 / (48)) + (606407.2542421686 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.4078613469363985 = (26062.367881727987 - 36692.200350321495) / 26062.367881727987 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.588650563362477 > 3.244491137293287. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.256128578514847 (T) = (0 -3687.403361054065) / Math.Sqrt((49666.461224947096 / (39)) + (62107.0850524346 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.14705377543802936 = (3214.673487863238 - 3687.403361054065) / 3214.673487863238 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Intersect(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 63.44445614601262 > 42.533671851537996. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.433203429770177 (T) = (0 -62702.31125756461) / Math.Sqrt((2494020.0098005105 / (51)) + (2778774.829866138 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4755086364005315 = (42495.38749601996 - 62702.31125756461) / 42495.38749601996 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.071042196046525 > 13.658043888648368. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.355585748342514 (T) = (0 -14251.697559256872) / Math.Sqrt((291491.6388248227 / (41)) + (498052.8077004989 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.10148554083988212 = (12938.615198153178 - 14251.697559256872) / 12938.615198153178 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 144.06003074845677 > 84.55828063710584. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -36.172836961311454 (T) = (0 -140726.12708347838) / Math.Sqrt((3854355.578777275 / (50)) + (10403579.233494595 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.726045665791319 = (81530.94085083861 - 140726.12708347838) / 81530.94085083861 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Range) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.00678122131449 > 24.4907743731737. IsChangePoint: Marked as a change because one of 9/19/2023 3:42:14 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.939514654214644 (T) = (0 -36733.10610632607) / Math.Sqrt((1803154.944619166 / (51)) + (1075336.610973507 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4273166602817443 = (25735.778981992098 - 36733.10610632607) / 25735.778981992098 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SkipHalfTakeHalf(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.142478805152013 > 21.190985479807896. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.708876762106673 (T) = (0 -29129.20275076581) / Math.Sqrt((451303.9480062207 / (51)) + (600752.3804812928 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.45116168199281786 = (20073.023641834265 - 29129.20275076581) / 20073.023641834265 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.0337373294679395 > 3.533903643299506. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.138472177804148 (T) = (0 -3930.0844677136115) / Math.Sqrt((69193.78319642779 / (40)) + (92785.58885574824 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.16657343291734433 = (3368.913054950456 - 3930.0844677136115) / 3368.913054950456 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IOrderedEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.61644287736646 > 22.399106047149008. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -42.6608042676181 (T) = (0 -31961.812317655633) / Math.Sqrt((588771.3121040417 / (48)) + (230026.64963264205 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.5445591287947459 = (20693.16203038219 - 31961.812317655633) / 20693.16203038219 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.GroupBy(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 48.572961777605784 > 37.28593647679438. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.21243312056073 (T) = (0 -49542.85212505012) / Math.Sqrt((1840502.2386760355 / (48)) + (1084703.2696849143 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.33233150422967245 = (37185.078914496444 - 49542.85212505012) / 37185.078914496444 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.5577632269061827 > 3.36195928127845. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 2:54:53 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.222879369168745 (T) = (0 -3728.3847415477694) / Math.Sqrt((75999.8658569623 / (39)) + (24105.0211345211 / (13))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (13) - 2, .025) and -0.1356764213116515 = (3282.9639425300957 - 3728.3847415477694) / 3282.9639425300957 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.587355902011282 > 8.397343125318681. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.877778586745357 (T) = (0 -11115.791848976802) / Math.Sqrt((59086.135047270356 / (51)) + (200084.13184289835 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.39361386174749474 = (7976.2351352033575 - 11115.791848976802) / 7976.2351352033575 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Distinct(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 40.83852018900344 > 28.711216911932862. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -49.615875519998745 (T) = (0 -40565.84596661838) / Math.Sqrt((1563401.6778637765 / (51)) + (98265.86390815648 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.40334709285271664 = (28906.49517373235 - 40565.84596661838) / 28906.49517373235 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Zip(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 65.8320785801964 > 44.64310059144376. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.897515723653584 (T) = (0 -66557.8139966394) / Math.Sqrt((3611182.3922664574 / (48)) + (2095227.4670445535 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.4801878503277676 = (44965.788620613974 - 66557.8139966394) / 44965.788620613974 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Append(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 57.59587962962962 > 41.442765578391345. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.08556753091047 (T) = (0 -56816.63820664982) / Math.Sqrt((1939851.7342994034 / (49)) + (902638.8002952867 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.41012771436892587 = (40291.83855313202 - 56816.63820664982) / 40291.83855313202 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AppendPrepend(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.90526390865955 > 39.47167184382005. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.09257903898907 (T) = (0 -56571.271235419605) / Math.Sqrt((1332098.0685740465 / (50)) + (3056727.4312971737 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.48593301276946815 = (38071.2123287325 - 56571.271235419605) / 38071.2123287325 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Skip_One(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 44.60085978835979 > 27.38856383898073. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.6694369497474 (T) = (0 -45039.684624283975) / Math.Sqrt((863283.4853734003 / (49)) + (4628641.965611767 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.7397714552589492 = (25888.276582615978 - 45039.684624283975) / 25888.276582615978 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.01214413711289 > 25.558194615066228. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.16273381294369 (T) = (0 -37005.92563166397) / Math.Sqrt((2433776.4514426277 / (51)) + (1084246.790660447 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4131690157014257 = (26186.482452204134 - 37005.92563166397) / 26186.482452204134 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.7429102233367 > 20.556257185034514. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.859598857067304 (T) = (0 -29424.84741596278) / Math.Sqrt((375019.30648682645 / (51)) + (463125.3215764435 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4975420747840171 = (19648.76173526315 - 29424.84741596278) / 19648.76173526315 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.321913031193485 > 20.491717965418307. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.671744158585852 (T) = (0 -29299.034839399905) / Math.Sqrt((487800.09991073655 / (51)) + (484625.3447031826 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.5185399025551324 = (19294.214653234092 - 29299.034839399905) / 19294.214653234092 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Select(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.982183507351024 > 20.04558936273995. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -41.433861671848 (T) = (0 -29314.578539816124) / Math.Sqrt((541060.8524071439 / (51)) + (185019.60869243104 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.5084021316668105 = (19434.19325947452 - 29314.578539816124) / 19434.19325947452 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.ElementAt(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 641.7013134152994 > 617.9770864612199. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.34227240744542 (T) = (0 -662.4200660801408) / Math.Sqrt((575.3849787309966 / (40)) + (768.1091467968092 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.10160921445326186 = (601.3203751285846 - 662.4200660801408) / 601.3203751285846 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Where(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.877273670708956 > 20.49590227320937. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -30.01515099544161 (T) = (0 -29739.371409760548) / Math.Sqrt((670129.5655333118 / (51)) + (392767.88206326665 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.5077459486261888 = (19724.391524220733 - 29739.371409760548) / 19724.391524220733 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.7371928257207836 > 3.4480265826239695. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.413588730844547 (T) = (0 -3695.967785277001) / Math.Sqrt((18397.0642580543 / (40)) + (14225.593360548532 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.10846736213183071 = (3334.304564609668 - 3695.967785277001) / 3334.304564609668 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.171158813996923 > 4.687636447140868. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.729399778741045 (T) = (0 -5040.292514365142) / Math.Sqrt((111646.64525699346 / (39)) + (36478.01551266873 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.09177236601587037 = (4616.61484688272 - 5040.292514365142) / 4616.61484688272 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.10818589786382 > 20.67806879807137. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -30.283290965395366 (T) = (0 -29664.37090830939) / Math.Sqrt((739492.836441703 / (49)) + (328374.7584493203 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4668133215134621 = (20223.685231943222 - 29664.37090830939) / 20223.685231943222 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.039823999885373 > 8.51648768450228. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.654393625421694 (T) = (0 -11162.51564773883) / Math.Sqrt((95383.24435445196 / (51)) + (54763.847185854036 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3593333800320427 = (8211.757183124351 - 11162.51564773883) / 8211.757183124351 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 142.7243611566186 > 88.07389590967098. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -73.39700586309381 (T) = (0 -140916.32499053856) / Math.Sqrt((5308906.855866136 / (51)) + (2125804.048679288 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.7100802681480753 = (82403.33954799872 - 140916.32499053856) / 82403.33954799872 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.472360096504516 > 8.608520573312038. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.764596903736063 (T) = (0 -11058.258889303806) / Math.Sqrt((240575.7366709494 / (52)) + (238693.7127718254 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.327739704689673 = (8328.634633915995 - 11058.258889303806) / 8328.634633915995 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.501658142389527 > 12.920579773364762. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 10:37:55 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.560828821742945 (T) = (0 -17301.027162161645) / Math.Sqrt((1359488.6062361342 / (50)) + (78493.27489592788 / (5))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (5) - 2, .025) and -0.22890106768890411 = (14078.454008261464 - 17301.027162161645) / 14078.454008261464 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.OrderByDescending(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 215.40741946347032 > 126.48668228944291. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -24.056579223748468 (T) = (0 -205266.2814547425) / Math.Sqrt((10794997.38947296 / (51)) + (46729497.215968676 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.6783753634071928 = (122300.58062699446 - 205266.2814547425) / 122300.58062699446 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.577860118038739 > 10.383694227555335. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 2:54:53 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -8.05126267860194 (T) = (0 -10654.365847060273) / Math.Sqrt((81302.7337924194 / (42)) + (89311.99702885175 / (12))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (12) - 2, .025) and -0.07896000624162879 = (9874.662439225083 - 10654.365847060273) / 9874.662439225083 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.88501350508779 > 8.488959773748554. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.771994626531598 (T) = (0 -10834.046542638762) / Math.Sqrt((72023.50637140354 / (51)) + (28176.467626221627 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3231258577312215 = (8188.22070427678 - 10834.046542638762) / 8188.22070427678 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 42.07745359078591 > 28.89398553665037. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -38.90414697938277 (T) = (0 -42556.88397765576) / Math.Sqrt((1480881.112586243 / (50)) + (365108.33637605 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.46596503645006454 = (29029.944725496447 - 42556.88397765576) / 29029.944725496447 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 44.28297622619047 > 29.810769813567937. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -36.70236601444275 (T) = (0 -44735.31526592581) / Math.Sqrt((2426558.085195251 / (50)) + (327427.8780295966 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.48319356594088797 = (30161.48147699605 - 44735.31526592581) / 30161.48147699605 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SelectToList(input: Range) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.08753622707529 > 24.202216475690204. IsChangePoint: Marked as a change because one of 9/18/2023 8:11:22 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.491092821461123 (T) = (0 -35617.30824449749) / Math.Sqrt((1632027.4359475705 / (50)) + (1556628.609072675 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.4329681288915013 = (24855.617878989335 - 35617.30824449749) / 24855.617878989335 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.581573674636175 > 13.376868558097277. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.948971065848039 (T) = (0 -14594.324025080312) / Math.Sqrt((732282.6878244624 / (13)) + (888359.3476733258 / (41))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (41) - 2, .025) and -0.15337121145431476 = (12653.622597947422 - 14594.324025080312) / 12653.622597947422 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.OrderBy(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 180.53089080459768 > 96.39466979656173. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.979989448478658 (T) = (0 -163321.6984176039) / Math.Sqrt((9951788.178472683 / (51)) + (132078932.51668848 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.7323376032242039 = (94278.2158129176 - 163321.6984176039) / 94278.2158129176 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Prepend(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.85424316206543 > 35.8755750234319. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -60.24469324827494 (T) = (0 -47676.78791229351) / Math.Sqrt((613579.3576488794 / (51)) + (154867.31912179515 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.39792891336448405 = (34105.302105488874 - 47676.78791229351) / 34105.302105488874 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Select(input: IList) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 39.33882092052923 > 24.78844615338115. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.818125040197165 (T) = (0 -36924.22618181848) / Math.Sqrt((1511828.0389695375 / (52)) + (3264772.398365989 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.46877629128844145 = (25139.448669496 - 36924.22618181848) / 25139.448669496 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.7261376878840724 > 3.2997881585693793. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.063866758183407 (T) = (0 -3627.319177997099) / Math.Sqrt((56596.686900091765 / (41)) + (49920.297021658655 / (13))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (13) - 2, .025) and -0.11218266459789437 = (3261.441931670948 - 3627.319177997099) / 3261.441931670948 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 953.4792467340919 > 831.0403509077438. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.852885087533757 (T) = (0 -954.7326370168043) / Math.Sqrt((695.295403697499 / (51)) + (485.3775237182028 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.22061723651583015 = (782.1720097464988 - 954.7326370168043) / 782.1720097464988 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Average(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.338599212831311 > 2.3050190059031515. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -4.200578578511119 (T) = (0 -2535.151235753744) / Math.Sqrt((4816.829620242058 / (38)) + (85203.41908012494 / (14))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (14) - 2, .025) and -0.15021967768331834 = (2204.0583072442696 - 2535.151235753744) / 2204.0583072442696 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.3810469520565345 > 4.013469142047674. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.032321652909785 (T) = (0 -4406.428556044247) / Math.Sqrt((31484.74019444757 / (41)) + (71173.56673156975 / (13))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (13) - 2, .025) and -0.14429073377069732 = (3850.7945804333012 - 4406.428556044247) / 3850.7945804333012 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.66950908739365 > 24.40659448699244. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.37300701453872 (T) = (0 -36875.14873724371) / Math.Sqrt((1752195.4816866303 / (51)) + (957232.8494257272 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.46497626490953897 = (25171.157799966622 - 36875.14873724371) / 25171.157799966622 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
610.97 ns 718.77 ns 1.18 0.11 False
803.99 ns 934.87 ns 1.16 0.11 False
1.42 μs 1.57 μs 1.11 0.12 False
859.24 ns 1.04 μs 1.21 0.13 False
1.98 μs 2.25 μs 1.14 0.41 False
3.33 μs 3.68 μs 1.10 0.24 False
623.30 ns 711.10 ns 1.14 0.08 False
433.05 ns 526.44 ns 1.22 0.10 False
867.40 ns 985.59 ns 1.14 0.18 False
1.97 μs 2.32 μs 1.18 0.39 False
816.59 ns 983.60 ns 1.20 0.13 False
2.15 μs 2.62 μs 1.22 0.05 False
1.89 μs 2.17 μs 1.15 0.37 False
1.22 μs 1.37 μs 1.12 0.06 False
815.38 ns 982.72 ns 1.21 0.20 False
803.28 ns 1.03 μs 1.28 0.16 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'
### Payloads [Baseline]() [Compare]() ### System.Reflection.Attributes.IsDefinedClassHitInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 718.7712781467 > 644.7267929617442. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -21.31970628153255 (T) = (0 -710.9536859360055) / Math.Sqrt((245.4513616399054 / (52)) + (45.91538693115182 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.13726023500823298 = (625.1459991748139 - 710.9536859360055) / 625.1459991748139 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodOverrideMiss #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 934.8672874098556 > 849.8927145431103. IsChangePoint: Marked as a change because one of 8/31/2023 1:23:15 PM, 9/21/2023 9:38:47 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.165914186337531 (T) = (0 -889.911836257299) / Math.Sqrt((402.9892332774793 / (22)) + (992.5597333454303 / (31))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (31) - 2, .025) and -0.08819510663977943 = (817.7870225912371 - 889.911836257299) / 817.7870225912371 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesClassMiss #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.57008314755079 > 1.4850448831375684. IsChangePoint: Marked as a change because one of 8/22/2023 3:00:25 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.197523823645344 (T) = (0 -1547.129218962365) / Math.Sqrt((1578.9514353264037 / (49)) + (397.87515689258163 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.07321702132183376 = (1441.5809554127595 - 1547.129218962365) / 1441.5809554127595 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodOverrideHit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.037579984995918 > 922.3621358354543. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.923164134863644 (T) = (0 -1014.0184338844558) / Math.Sqrt((660.8530807792829 / (50)) + (1785.2073000546397 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.14313924983241244 = (887.0471677296654 - 1014.0184338844558) / 887.0471677296654 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesMethodOverrideMiss #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.254950019981491 > 2.1869085798256744. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.333474163513746 (T) = (0 -2304.873276874243) / Math.Sqrt((6089.330904064179 / (49)) + (5022.626007508424 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.09404284878172736 = (2106.7486336945917 - 2304.873276874243) / 2106.7486336945917 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesMethodBaseHitInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.677216753636939 > 3.4565837430047996. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 4:14:13 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.787761210664045 (T) = (0 -3531.482609824832) / Math.Sqrt((9608.388664752438 / (44)) + (12074.68594253818 / (10))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (10) - 2, .025) and -0.07825780320497824 = (3275.1746375755115 - 3531.482609824832) / 3275.1746375755115 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedClassHit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 711.0957521833698 > 653.4941497185288. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -24.99175840574524 (T) = (0 -708.4536554051042) / Math.Sqrt((198.9035461885404 / (50)) + (27.839507553392366 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.13207771032807525 = (625.7994914499243 - 708.4536554051042) / 625.7994914499243 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedClassMiss #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 526.4352008195274 > 454.8056423141701. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.162386247654094 (T) = (0 -519.9757691475002) / Math.Sqrt((86.38802398733253 / (51)) + (28.527616517506726 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.19175676283883145 = (436.3103154614275 - 519.9757691475002) / 436.3103154614275 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodOverrideHitInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 985.589623810468 > 917.4118056052125. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.620782283896613 (T) = (0 -992.5398721246866) / Math.Sqrt((509.84712891288734 / (51)) + (95.8497985158516 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.12991967059325937 = (878.4163139699638 - 992.5398721246866) / 878.4163139699638 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesMethodBaseHit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3209645593616477 > 2.149020090991288. IsChangePoint: Marked as a change because one of 9/26/2023 6:21:18 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.644632816793429 (T) = (0 -2399.758558686289) / Math.Sqrt((7642.037537329512 / (46)) + (7667.291154715471 / (6))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (6) - 2, .025) and -0.1377247003344496 = (2109.2611929589357 - 2399.758558686289) / 2109.2611929589357 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodOverrideMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 983.6046129536496 > 860.1409701053609. IsChangePoint: Marked as a change because one of 8/31/2023 1:23:15 PM, 9/20/2023 11:58:56 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.434986382264032 (T) = (0 -969.7463793619531) / Math.Sqrt((915.0783439589525 / (50)) + (459.3267628287231 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.14175929886191282 = (849.3439732249876 - 969.7463793619531) / 849.3439732249876 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedClassMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.617256605630796 > 2.2798635194266836. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/19/2023 8:08:41 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.168630208293237 (T) = (0 -2568.5933502963208) / Math.Sqrt((4555.958563601094 / (51)) + (1446.9735723471529 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.13270045279248474 = (2267.6722199270607 - 2568.5933502963208) / 2267.6722199270607 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesMethodOverrideHit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.166712367683686 > 1.9929177702278458. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -4.969825769629921 (T) = (0 -2094.759680832416) / Math.Sqrt((5759.445570184245 / (49)) + (5002.679153883586 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.09619951024280045 = (1910.9292252542984 - 2094.759680832416) / 1910.9292252542984 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodBaseMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3657321424603175 > 1.2725831361455928. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.8411249706479085 (T) = (0 -1353.9707882924895) / Math.Sqrt((490.6896476208479 / (48)) + (1141.1816023044312 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.11056144813499294 = (1219.1768321926381 - 1353.9707882924895) / 1219.1768321926381 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodBaseHitInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 982.7222993626139 > 865.3664417803536. IsChangePoint: Marked as a change because one of 8/31/2023 1:23:15 PM, 9/21/2023 9:38:47 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -8.048764803389094 (T) = (0 -962.2395194824805) / Math.Sqrt((852.89279427919 / (51)) + (721.0929964229307 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.1330186418108214 = (849.2706862656761 - 962.2395194824805) / 849.2706862656761 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodBaseHit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0276859199313855 > 846.8908787768079. IsChangePoint: Marked as a change because one of 8/31/2023 1:23:15 PM, 9/21/2023 9:38:47 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.982096924338995 (T) = (0 -1000.2480310242106) / Math.Sqrt((1260.911368708688 / (51)) + (1301.5962860282866 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.175530887162447 = (850.8904716563062 - 1000.2480310242106) / 850.8904716563062 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Exceptions.Handling

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.03 μs 9.59 μs 1.19 0.17 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Exceptions.Handling*'
### Payloads [Baseline]() [Compare]() ### Exceptions.Handling.ThrowAndCatchFinally(kind: Hardware) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.585733216404925 > 8.45044311593029. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.900419702815366 (T) = (0 -9634.261139277602) / Math.Sqrt((27228.231094830873 / (49)) + (32138.801002079228 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.19415358342724998 = (8067.857663356029 - 9634.261139277602) / 8067.857663356029 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Perf_Frozen<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.56 μs 11.57 μs 1.77 0.38 True
1.75 μs 2.45 μs 1.40 0.21 True
58.58 μs 96.82 μs 1.65 0.47 True
7.66 μs 12.01 μs 1.57 0.32 True
1.15 μs 1.34 μs 1.17 0.16 False
1.55 μs 2.19 μs 1.41 0.17 True
55.81 μs 91.94 μs 1.65 0.36 True

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.572442816574618 > 7.2555965391046655. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -44.97365835271446 (T) = (0 -11516.828075270441) / Math.Sqrt((111982.89421773063 / (51)) + (33023.98673035151 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.6645214945114573 = (6919.002315828111 - 11516.828075270441) / 6919.002315828111 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4458512910042547 > 1.7655462597622815. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -20.035297993927145 (T) = (0 -2370.620983140403) / Math.Sqrt((3057.454554053837 / (49)) + (4769.6053433784 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.42730425172034553 = (1660.9079530752238 - 2370.620983140403) / 1660.9079530752238 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 96.81638734567902 > 61.77648856085526. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -30.717806331096543 (T) = (0 -96886.4846182898) / Math.Sqrt((5905299.761878484 / (48)) + (5176572.018931759 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.6062486016860138 = (60318.48651360194 - 96886.4846182898) / 60318.48651360194 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.007255269979767 > 8.11964167790447. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -65.64998694819526 (T) = (0 -12107.890568994171) / Math.Sqrt((110336.33759884023 / (48)) + (9702.593644850462 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.5940823733753275 = (7595.523776702198 - 12107.890568994171) / 7595.523776702198 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.ToFrozenSet(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3412089160514895 > 1.2335015363240258. IsChangePoint: Marked as a change because one of 9/25/2023 10:28:34 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.464789680501509 (T) = (0 -1327.4049328894014) / Math.Sqrt((1362.5090954666325 / (47)) + (1022.2447069962608 / (8))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (8) - 2, .025) and -0.1332401659300789 = (1171.3359381327316 - 1327.4049328894014) / 1171.3359381327316 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1902527902028646 > 1.6542912137252583. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.07179641931128 (T) = (0 -2202.456938148133) / Math.Sqrt((3183.628367399257 / (52)) + (1928.216206584918 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.3810522970385845 = (1594.7672241456034 - 2202.456938148133) / 1594.7672241456034 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.Contains_True(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 91.94165493393889 > 60.55883267666586. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -92.63527899206856 (T) = (0 -92238.4704033248) / Math.Sqrt((6560903.349016654 / (49)) + (75461.23303886007 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.6461672540393262 = (56032.259284099004 - 92238.4704033248) / 56032.259284099004 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.20 μs 4.76 μs 1.13 0.16 False
2.24 μs 2.92 μs 1.30 0.12 False
1.98 μs 2.48 μs 1.25 0.19 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.759936389312976 > 4.4125286362806895. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.245156214045533 (T) = (0 -4838.974900428409) / Math.Sqrt((15431.850797896708 / (49)) + (6408.081091886213 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.13615990759127064 = (4259.061482540195 - 4838.974900428409) / 4259.061482540195 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.923174176604202 > 2.351735359787945. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -24.113325805869888 (T) = (0 -2956.3214726733536) / Math.Sqrt((7373.082560418133 / (49)) + (2426.2426782906628 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2893584522792629 = (2292.8623668982955 - 2956.3214726733536) / 2292.8623668982955 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4782372230038194 > 2.09051685319042. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.860883129170753 (T) = (0 -2425.4871395201662) / Math.Sqrt((3789.6635423805396 / (50)) + (4839.6426567005665 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.21262543214796145 = (2000.194846007662 - 2425.4871395201662) / 2000.194846007662 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.84 μs 2.11 μs 1.14 0.22 False
2.91 μs 3.94 μs 1.35 0.15 False
1.69 μs 2.16 μs 1.28 0.19 True
1.43 μs 1.95 μs 1.36 0.14 True
1.81 μs 2.32 μs 1.28 0.16 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.106913475656443 > 1.9647598040213863. IsChangePoint: Marked as a change because one of 8/22/2023 10:50:09 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.99801433554254 (T) = (0 -2193.039448934367) / Math.Sqrt((3990.193031171009 / (50)) + (6522.677342307518 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.12754156853651827 = (1944.9743673581822 - 2193.039448934367) / 1944.9743673581822 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.937913406822692 > 3.0301876039838493. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -25.95294931610033 (T) = (0 -3961.5145027102294) / Math.Sqrt((9490.262539856054 / (51)) + (4469.798504494782 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.36650029897637926 = (2899.022053400009 - 3961.5145027102294) / 2899.022053400009 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.156437410154283 > 1.7576746529170868. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -45.53875979318735 (T) = (0 -2160.127484820386) / Math.Sqrt((2780.183091693671 / (48)) + (210.31960272524873 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.2846982838403234 = (1681.4278589702474 - 2160.127484820386) / 1681.4278589702474 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.95147577703793 > 1.496071655094317. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.05245497564219 (T) = (0 -1906.196629156383) / Math.Sqrt((1952.1934228152893 / (52)) + (2466.3053461570485 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.3434175507069774 = (1418.9159789919684 - 1906.196629156383) / 1418.9159789919684 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.322353847660471 > 1.9036254847550769. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.13483533958027 (T) = (0 -2304.5626461894494) / Math.Sqrt((3664.2932551040385 / (50)) + (1491.7506946776473 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2545344770389464 = (1836.9862991959092 - 2304.5626461894494) / 1836.9862991959092 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
110.93 μs 178.58 μs 1.61 0.28 True
27.48 μs 29.98 μs 1.09 0.40 False
383.36 μs 556.35 μs 1.45 0.19 True
58.79 μs 91.94 μs 1.56 0.24 True
142.04 μs 192.34 μs 1.35 0.18 True
141.76 μs 216.41 μs 1.53 0.30 True
225.56 μs 349.82 μs 1.55 0.14 True
126.19 μs 198.62 μs 1.57 0.15 False
370.81 μs 557.44 μs 1.50 0.15 True
134.44 μs 204.02 μs 1.52 0.22 True
100.95 μs 132.16 μs 1.31 0.17 True
57.31 μs 90.29 μs 1.58 0.25 False
30.37 μs 35.06 μs 1.15 0.29 False

graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateForEach<Int32>.ConcurrentQueue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 178.5807289772727 > 109.61339792373681. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -104.11675854002256 (T) = (0 -177242.04218362397) / Math.Sqrt((11407964.564557899 / (51)) + (1150078.1383228549 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.7241442500821381 = (102800.00769957628 - 177242.04218362397) / 102800.00769957628 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ImmutableQueue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.983937643129774 > 29.617240126727907. IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -4.582324872388262 (T) = (0 -30499.065844327968) / Math.Sqrt((8639737.71113302 / (41)) + (1035525.4272482971 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.08715187628339278 = (28054.098520801006 - 30499.065844327968) / 28054.098520801006 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 556.3469827586206 > 403.206449994992. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -71.02167879909696 (T) = (0 -555464.1806041813) / Math.Sqrt((115074248.5534856 / (50)) + (14365310.901194172 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4500776584170098 = (383058.2295920335 - 555464.1806041813) / 383058.2295920335 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ConcurrentStack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 91.9368809867878 > 60.12387336783275. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -33.01717454456801 (T) = (0 -89436.42248738334) / Math.Sqrt((4366399.475948696 / (48)) + (3494380.5616279966 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.5687969345119868 = (57009.559694993135 - 89436.42248738334) / 57009.559694993135 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ImmutableList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 192.33734751016263 > 140.73350163357924. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -20.905652542875 (T) = (0 -189716.10467249385) / Math.Sqrt((22010431.630812123 / (49)) + (26578493.092006654 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4154164905255676 = (134035.5336697039 - 189716.10467249385) / 134035.5336697039 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 216.4062500611546 > 151.50319082889732. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.25213998095413 (T) = (0 -215685.9253145624) / Math.Sqrt((26289245.132078554 / (49)) + (59710359.17941991 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4987511484927564 = (143910.43204969048 - 215685.9253145624) / 143910.43204969048 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 349.824722037037 > 235.8688399692029. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -38.80810213159348 (T) = (0 -353762.22158576094) / Math.Sqrt((48509166.634304225 / (51)) + (38186345.66292874 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.5514315540563605 = (228023.09303353867 - 353762.22158576094) / 228023.09303353867 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 198.62149780389367 > 128.89012846214553. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.086210196729382 (T) = (0 -202859.1827357678) / Math.Sqrt((8534775.776572902 / (50)) + (72426222.3685033 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.6420349789829553 = (123541.32849314505 - 202859.1827357678) / 123541.32849314505 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 557.4397987068967 > 389.61321829997763. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.529877192428895 (T) = (0 -538498.1014520903) / Math.Sqrt((86264978.90066133 / (50)) + (226981557.90826908 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4704190085099033 = (366220.85156379663 - 538498.1014520903) / 366220.85156379663 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 204.02204024122807 > 141.476378372252. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -34.750350215512455 (T) = (0 -208469.29285938013) / Math.Sqrt((19434459.09288797 / (50)) + (14704244.9510851 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.5061904098554171 = (138408.32573047097 - 208469.29285938013) / 138408.32573047097 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 132.16043139948286 > 102.13563582098585. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.510495326312356 (T) = (0 -132424.515841387) / Math.Sqrt((5352647.648055328 / (51)) + (11565698.268814998 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3423433001920542 = (98651.75013160979 - 132424.515841387) / 98651.75013160979 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.ConcurrentBag(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 90.2863554238506 > 60.28118553068368. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.11177417926276 (T) = (0 -90351.60112789435) / Math.Sqrt((7345892.727490075 / (51)) + (2701165.3900959147 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.5703958063554887 = (57534.28579103169 - 90351.60112789435) / 57534.28579103169 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<Int32>.SortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.06131159248055 > 32.17791920562094. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/23/2023 6:58:52 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.193879888390581 (T) = (0 -33890.21301588558) / Math.Sqrt((2690854.919620121 / (30)) + (3905918.588225466 / (23))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (23) - 2, .025) and -0.08468462928294276 = (31244.300970955523 - 33890.21301588558) / 31244.300970955523 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
102.03 μs 126.62 μs 1.24 0.06 False
211.83 μs 242.71 μs 1.15 0.06 False
91.60 μs 105.89 μs 1.16 0.07 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 126.62226432891704 > 106.38004868827036. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.31224256759681 (T) = (0 -126849.79405298803) / Math.Sqrt((6623399.442737236 / (52)) + (7112720.169396701 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.23212957272773063 = (102951.66747127383 - 126849.79405298803) / 102951.66747127383 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 242.71163511439732 > 217.78174943333332. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.060283591935336 (T) = (0 -236588.27907447005) / Math.Sqrt((30341610.828377966 / (49)) + (30931339.956498314 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.12444690574042208 = (210404.13546132014 - 236588.27907447005) / 210404.13546132014 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 105.89266546803651 > 97.57432297252495. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.726090785563162 (T) = (0 -107961.92332552525) / Math.Sqrt((4731914.26886648 / (51)) + (2408124.6295307465 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.15856061929445928 = (93186.25329356693 - 107961.92332552525) / 93186.25329356693 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
24.35 μs 27.79 μs 1.14 0.02 False
22.38 μs 25.22 μs 1.13 0.05 False
5.61 μs 6.36 μs 1.13 0.07 False
24.39 μs 28.43 μs 1.17 0.04 False
24.06 μs 28.35 μs 1.18 0.05 False
24.44 μs 28.14 μs 1.15 0.07 False
5.70 μs 6.78 μs 1.19 0.09 False
22.09 μs 25.45 μs 1.15 0.07 False
22.75 μs 26.42 μs 1.16 0.05 False
21.78 μs 25.50 μs 1.17 0.07 False
22.17 μs 26.18 μs 1.18 0.06 False
5.51 μs 6.74 μs 1.22 0.11 False
22.14 μs 25.46 μs 1.15 0.07 False
22.31 μs 26.14 μs 1.17 0.07 False
21.44 μs 25.00 μs 1.17 0.06 False
5.52 μs 6.88 μs 1.25 0.10 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.793241265597153 > 24.815777103470996. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.00433542662638 (T) = (0 -27843.743636851283) / Math.Sqrt((339963.35571846773 / (51)) + (144270.35034577487 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.15429452606679359 = (24121.87098532606 - 27843.743636851283) / 24121.87098532606 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.216172462406018 > 23.70169370871509. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.374801230335828 (T) = (0 -25707.907886372006) / Math.Sqrt((373319.0524224013 / (51)) + (155951.81697790985 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.14770717107509934 = (22399.361556912176 - 25707.907886372006) / 22399.361556912176 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.360815299027609 > 5.809815045378254. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.371562350308054 (T) = (0 -6473.242834404913) / Math.Sqrt((12981.78488610656 / (48)) + (37371.01152897163 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.16542829720932803 = (5554.389617881591 - 6473.242834404913) / 5554.389617881591 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.425902482355795 > 25.475308316199378. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.27690291676403 (T) = (0 -28106.327414313353) / Math.Sqrt((300512.9429845869 / (48)) + (54591.02313564389 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.15195872139743125 = (24398.72791641163 - 28106.327414313353) / 24398.72791641163 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.347680885143856 > 25.617077537774723. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -36.24050303538282 (T) = (0 -28254.19642870491) / Math.Sqrt((302028.7723830308 / (51)) + (25687.74297952998 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.16619216194425526 = (24227.736517796522 - 28254.19642870491) / 24227.736517796522 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.142742257855808 > 25.01277562770642. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -38.387297704474925 (T) = (0 -28165.00555135112) / Math.Sqrt((272518.25526140386 / (50)) + (14901.47141673362 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.1615892659037826 = (24246.957490122073 - 28165.00555135112) / 24246.957490122073 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.7754387785522345 > 5.851163153290112. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.786774028391198 (T) = (0 -6606.221099176077) / Math.Sqrt((13640.475593221016 / (51)) + (20230.524856546705 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.179649532154155 = (5600.155740419337 - 6606.221099176077) / 5600.155740419337 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.451132436528983 > 23.163352869810026. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.900249739543842 (T) = (0 -25067.954614772876) / Math.Sqrt((235433.5281297305 / (52)) + (88112.19490735825 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.12342288183134306 = (22313.907808169668 - 25067.954614772876) / 22313.907808169668 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.415682186447814 > 23.90881519781676. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -8.410225431503974 (T) = (0 -26207.793974751028) / Math.Sqrt((307838.6423273525 / (51)) + (535177.3141121513 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.15984161682278164 = (22596.01103687199 - 26207.793974751028) / 22596.01103687199 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.500542997643677 > 23.443858039837. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -21.824166688241096 (T) = (0 -25851.505788909388) / Math.Sqrt((340000.24785143766 / (51)) + (63188.56292999536 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.14485902880239032 = (22580.51440267893 - 25851.505788909388) / 22580.51440267893 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.175021080508476 > 23.353723312455422. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.301906381381361 (T) = (0 -26195.166060471773) / Math.Sqrt((315633.4541600281 / (50)) + (214007.97094086066 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.154107126872073 = (22697.343643884607 - 26195.166060471773) / 22697.343643884607 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "Dynamic") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.737052315211188 > 5.82659881786537. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.087390536988973 (T) = (0 -6615.6633434607675) / Math.Sqrt((14962.193994417888 / (48)) + (16824.441836333095 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.18103773492505498 = (5601.568136077021 - 6615.6633434607675) / 5601.568136077021 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "ILEmit") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.46043023208469 > 23.26997549101484. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.58380279249165 (T) = (0 -26026.363308950535) / Math.Sqrt((248943.19401731424 / (49)) + (278078.8945834648 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.16625351354106546 = (22316.214276540406 - 26026.363308950535) / 22316.214276540406 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Runtime") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.13536697322919 > 23.848044970120736. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -30.07903678764601 (T) = (0 -25855.87088015755) / Math.Sqrt((175549.49384925357 / (50)) + (35396.503208059185 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.1485473569548479 = (22511.802167835212 - 25855.87088015755) / 22511.802167835212 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Transient(Mode: "Expressions") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.996163826669157 > 23.611397319498636. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.344390871591274 (T) = (0 -25271.382543371135) / Math.Sqrt((286317.477354281 / (51)) + (271295.1282435661 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.13849833675777368 = (22197.11854418621 - 25271.382543371135) / 22197.11854418621 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.TimeToFirstService.BuildProvider(Mode: "ILEmit") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.8808521784834 > 5.799649055984982. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.48587761592401 (T) = (0 -6661.485511011651) / Math.Sqrt((13716.412465403728 / (49)) + (29397.66954628477 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.19576628198761206 = (5570.892582736886 - 6661.485511011651) / 5570.892582736886 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Activator<EmptyClass>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
442.58 ns 564.75 ns 1.28 0.19 True
312.21 ns 452.56 ns 1.45 0.21 True
12.06 μs 13.63 μs 1.13 0.42 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Activator&lt;EmptyClass&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Reflection.Activator<EmptyClass>.CreateInstanceType #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 564.7521106400574 > 457.378104295875. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.66315385616183 (T) = (0 -575.2777707150733) / Math.Sqrt((174.51176330907663 / (51)) + (265.1617568921862 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3189886049975192 = (436.1506752487489 - 575.2777707150733) / 436.1506752487489 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Activator<EmptyClass>.CreateInstanceGeneric #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 452.5568543277151 > 324.54570750997357. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -68.70649165643678 (T) = (0 -448.51810795761935) / Math.Sqrt((79.12442796958112 / (50)) + (10.210965512541025 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4524473086825635 = (308.8016379502578 - 448.51810795761935) / 308.8016379502578 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Activator<EmptyClass>.CreateInstanceNames #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.630619256315333 > 12.635636679397205. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.176875960099711 (T) = (0 -14293.837530316992) / Math.Sqrt((649015.3640273053 / (47)) + (279727.2720696842 / (4))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (4) - 2, .025) and -0.14292415315317195 = (12506.374540149705 - 14293.837530316992) / 12506.374540149705 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
668.48 μs 1.39 ms 2.09 0.30 True
580.70 μs 1.01 ms 1.73 0.22 True
86.41 μs 183.00 μs 2.12 0.40 True
183.40 μs 284.12 μs 1.55 0.13 True
82.75 μs 119.25 μs 1.44 0.28 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrueComparer<Int32>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.394021526041667 > 705.2975073556386. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.908807777088384 (T) = (0 -1279778.1472842263) / Math.Sqrt((1025494994.4255087 / (51)) + (6062564005.958934 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.8400243676726515 = (695522.3907729816 - 1279778.1472842263) / 695522.3907729816 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<Int32>.SortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0063278658854167 > 591.9114279477164. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -55.07579024559061 (T) = (0 -1011374.6532664764) / Math.Sqrt((551086521.5706757 / (49)) + (197347828.18909714 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.7357005645197306 = (582689.5917074984 - 1011374.6532664764) / 582689.5917074984 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<Int32>.HashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 182.99608681797847 > 91.83700494908211. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.344319228776826 (T) = (0 -175563.79534156574) / Math.Sqrt((20624982.981831513 / (50)) + (38825524.04532427 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.9132511981055097 = (91762.02033241009 - 175563.79534156574) / 91762.02033241009 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<Int32>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 284.1193391262755 > 199.06266545474764. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -55.81258892341038 (T) = (0 -283676.95658639167) / Math.Sqrt((29280752.446328178 / (50)) + (7967544.398780599 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.46170919221696893 = (194072.0890973805 - 283676.95658639167) / 194072.0890973805 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<Int32>.FrozenSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 119.24939076028137 > 88.23624518107826. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.28790698904108 (T) = (0 -120904.44939821848) / Math.Sqrt((8984454.744045306 / (50)) + (6474325.890622647 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4340213276068467 = (84311.47227077073 - 120904.44939821848) / 84311.47227077073 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
281.34 μs 364.01 μs 1.29 0.10 True
292.99 μs 400.85 μs 1.37 0.14 True
4.41 ms 6.29 ms 1.43 0.11 True
44.70 μs 57.83 μs 1.29 0.08 True
27.55 μs 35.06 μs 1.27 0.06 False
10.03 μs 13.50 μs 1.35 0.11 True
670.16 μs 874.12 μs 1.30 0.09 True
9.75 ms 14.20 ms 1.46 0.09 True
8.06 μs 10.56 μs 1.31 0.11 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 364.0141997987478 > 286.6527509896779. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.889199354678087 (T) = (0 -355809.83647814114) / Math.Sqrt((45654829.03035784 / (51)) + (33789503.880630076 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.30037310993977345 = (273621.34279646893 - 355809.83647814114) / 273621.34279646893 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 1000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 400.84874260355025 > 306.4639300954861. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.712937329515682 (T) = (0 -399542.7582573537) / Math.Sqrt((44485393.569683425 / (50)) + (55192393.287977375 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.36207527858901756 = (293333.83002974885 - 399542.7582573537) / 293333.83002974885 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 1000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.2943640991453 > 4.587284718073918. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -25.957421540168664 (T) = (0 -6130578.450976801) / Math.Sqrt((10702746110.709892 / (52)) + (17367394648.082836 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.399642570592552 = (4380102.877537772 - 6130578.450976801) / 4380102.877537772 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 57.83498753063726 > 45.469751931138354. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -42.81555986765169 (T) = (0 -57952.95503219914) / Math.Sqrt((711071.3145724268 / (51)) + (389236.4685036414 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.32700252205272023 = (43672.076027822906 - 57952.95503219914) / 43672.076027822906 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.06470581636501 > 28.85838333775506. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -31.77514203312454 (T) = (0 -35497.22472341288) / Math.Sqrt((344300.73184804973 / (50)) + (156422.23026355487 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.27792117108708464 = (27777.31954563096 - 35497.22472341288) / 27777.31954563096 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.499484137806634 > 10.75926405080899. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -44.10172654128589 (T) = (0 -13641.823535873253) / Math.Sqrt((68008.12777686633 / (49)) + (17097.378910115207 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.32146525107844004 = (10323.255586736193 - 13641.823535873253) / 10323.255586736193 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 874.1215233796297 > 706.3615296744356. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.677069299396564 (T) = (0 -876523.6209666964) / Math.Sqrt((235357763.9882868 / (51)) + (148881698.30377856 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.31774835581924177 = (665167.6832651127 - 876523.6209666964) / 665167.6832651127 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Dequeue_And_Enqueue(Size: 1000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.196946078431374 > 10.206123382562499. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -21.485696382577153 (T) = (0 -13883235.194831058) / Math.Sqrt((33237724594.01363 / (48)) + (141037955342.507 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.4153088007888271 = (9809332.90819162 - 13883235.194831058) / 9809332.90819162 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.562970084433228 > 8.425393652727502. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -24.921410735700302 (T) = (0 -10721.941499424347) / Math.Sqrt((66680.40284531409 / (51)) + (27036.277295659347 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.3091031817050305 = (8190.295195417403 - 10721.941499424347) / 8190.295195417403 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.89 μs 3.31 μs 1.14 0.20 False
2.68 μs 3.27 μs 1.22 0.14 False
2.92 μs 3.86 μs 1.32 0.13 False
2.32 μs 2.99 μs 1.29 0.18 True
2.98 μs 3.62 μs 1.22 0.14 False
3.35 μs 4.01 μs 1.20 0.09 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.313105085569446 > 2.8545623857804614. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -8.974612100332093 (T) = (0 -3394.60416723701) / Math.Sqrt((6901.269965046071 / (50)) + (19582.872251850913 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.23088097318584502 = (2757.865497303836 - 3394.60416723701) / 2757.865497303836 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2745644355670103 > 2.7195455971374947. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -21.12400126337315 (T) = (0 -3276.662332827777) / Math.Sqrt((4393.75161681264 / (48)) + (3841.6240962540555 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.26437193329792086 = (2591.5335879697236 - 3276.662332827777) / 2591.5335879697236 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.8594632433773053 > 3.1255916564075497. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.93661919319168 (T) = (0 -3748.9589365623297) / Math.Sqrt((8270.41118307464 / (51)) + (12330.983923231228 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.24465948159370468 = (3012.0358154200007 - 3748.9589365623297) / 3012.0358154200007 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9857343557996656 > 2.4229674756207253. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.03233394723909 (T) = (0 -2978.84548050223) / Math.Sqrt((4112.050493442509 / (51)) + (5009.227693746491 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2837106637630339 = (2320.496015644308 - 2978.84548050223) / 2320.496015644308 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.6190479662238273 > 3.173021887178539. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -24.090959238799083 (T) = (0 -3601.2754790565346) / Math.Sqrt((7826.447303325559 / (50)) + (1242.2892858473597 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.19019099420389468 = (3025.7962768953626 - 3601.2754790565346) / 3025.7962768953626 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.011814827896225 > 3.3831623565500175. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.49234173062742 (T) = (0 -3960.531358159432) / Math.Sqrt((10322.340495472106 / (51)) + (4527.547542047779 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2056245448864972 = (3285.0453940719112 - 3960.531358159432) / 3285.0453940719112 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.IO.Tests.Perf_RandomAccess

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
192.16 μs 218.36 μs 1.14 0.06 False
191.01 μs 213.59 μs 1.12 0.07 False
9.84 ms 10.62 ms 1.08 0.02 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_RandomAccess*'
### Payloads [Baseline]() [Compare]() ### System.IO.Tests.Perf_RandomAccess.ReadScatter(fileSize: 1048576, buffersSize: 16384, options: Asynchronous) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 218.35607409037559 > 203.10640819818042. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.293555560545567 (T) = (0 -220799.57563447353) / Math.Sqrt((19043963.96190519 / (49)) + (9513242.881816676 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.13992510057296423 = (193696.5643826005 - 220799.57563447353) / 193696.5643826005 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.Perf_RandomAccess.ReadScatter(fileSize: 1048576, buffersSize: 16384, options: None) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 213.59477166754476 > 201.9623301941069. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.797113185277902 (T) = (0 -221329.46964337962) / Math.Sqrt((20450157.928195324 / (51)) + (67311803.85858257 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.14609489704730955 = (193116.18105410985 - 221329.46964337962) / 193116.18105410985 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.Perf_RandomAccess.ReadScatter(fileSize: 104857600, buffersSize: 65536, options: Asynchronous) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.62420130844156 > 10.323642967272729. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.421935208832329 (T) = (0 -10512958.25959596) / Math.Sqrt((15451410294.59842 / (48)) + (14616308917.51422 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.06667740806977646 = (9855799.119829355 - 10512958.25959596) / 9855799.119829355 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.41 μs 6.51 μs 1.20 0.09 True
2.30 μs 2.55 μs 1.11 0.18 False
5.16 μs 6.53 μs 1.27 0.13 True
5.15 μs 6.44 μs 1.25 0.10 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.CaseInsensitiveNotMatching #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.510755545922656 > 5.567852450746095. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.4899578971109 (T) = (0 -6525.714498550347) / Math.Sqrt((16906.416838370562 / (50)) + (7711.281448847515 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.23951315774190293 = (5264.739997144235 - 6525.714498550347) / 5264.739997144235 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.MissingProperties #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.5489734967408677 > 2.323054095278499. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.439531999848477 (T) = (0 -2569.703476146654) / Math.Sqrt((5203.023874355299 / (51)) + (7059.008604423788 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.14296931667603968 = (2248.269869238322 - 2569.703476146654) / 2248.269869238322 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.Baseline #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.528738219935259 > 5.412455804201913. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.5337086820953 (T) = (0 -6606.57412976045) / Math.Sqrt((21289.55928350008 / (50)) + (9215.264900970682 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2655536686827024 = (5220.303408102118 - 6606.57412976045) / 5220.303408102118 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.CaseInsensitiveMatching #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.441923351016345 > 5.500424815534093. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.704047363966573 (T) = (0 -6547.422052509661) / Math.Sqrt((24317.7136223795 / (51)) + (18038.661076541055 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.23949641059505863 = (5282.324334740403 - 6547.422052509661) / 5282.324334740403 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.12 ms 1.39 ms 1.24 0.04 False
1.02 ms 1.25 ms 1.22 0.11 False
1.27 ms 1.56 ms 1.23 0.06 False
1.09 ms 1.31 ms 1.20 0.08 False
1.23 ms 1.44 ms 1.17 0.06 False
1.01 ms 1.24 ms 1.23 0.06 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3858915662878788 > 1.183855398954327. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.011588497777645 (T) = (0 -1359725.184422912) / Math.Sqrt((660296809.7697545 / (50)) + (646246884.357536 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.21229497354360322 = (1121612.4904389915 - 1359725.184422912) / 1121612.4904389915 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2453368787774726 > 1.0749692384667968. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -20.74426878299501 (T) = (0 -1229600.3695217844) / Math.Sqrt((928952795.1493777 / (51)) + (232011762.5828806 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17272642320384013 = (1048497.1986582915 - 1229600.3695217844) / 1048497.1986582915 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5623083250000005 > 1.3585479685708675. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.181485731613837 (T) = (0 -1563733.3317463498) / Math.Sqrt((520701880.77929527 / (50)) + (456467877.5761985 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.19822350891366555 = (1305043.1076619946 - 1563733.3317463498) / 1305043.1076619946 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3101450252403846 > 1.1591295421995194. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.881177413278794 (T) = (0 -1320903.955284299) / Math.Sqrt((530589413.6365857 / (51)) + (1188693419.5893936 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.19920508068926288 = (1101482.9544627077 - 1320903.955284299) / 1101482.9544627077 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4407784025349653 > 1.2705378611177884. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -34.38504303913468 (T) = (0 -1442913.068958819) / Math.Sqrt((904178241.8911017 / (50)) + (47820328.88073273 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.1614424361978853 = (1242345.7452461957 - 1442913.068958819) / 1242345.7452461957 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2365788506410258 > 1.058057111904297. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.507599088390126 (T) = (0 -1242236.9640536946) / Math.Sqrt((596455927.5076629 / (50)) + (330757526.4188127 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.21395075424297613 = (1023300.9532815504 - 1242236.9640536946) / 1023300.9532815504 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.06 ms 1.21 ms 1.13 0.06 False
1.12 ms 1.38 ms 1.24 0.11 False
2.36 ms 2.71 ms 1.15 0.08 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.205628699704142 > 1.1157571994535256. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.407896318209167 (T) = (0 -1231510.5078582878) / Math.Sqrt((736722471.2457913 / (48)) + (465823848.61204654 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.16773080421196004 = (1054618.4989008398 - 1231510.5078582878) / 1054618.4989008398 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3826458307291665 > 1.1695047302187502. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -43.12980935356808 (T) = (0 -1370313.7884610617) / Math.Sqrt((640310582.4494108 / (51)) + (85161145.39882869 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.22415202377622873 = (1119398.376873125 - 1370313.7884610617) / 1119398.376873125 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.7091201395833333 > 2.485112866119792. IsChangePoint: Marked as a change because one of 8/28/2023 11:06:53 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.196496708588734 (T) = (0 -2743075.540344551) / Math.Sqrt((3576559958.618361 / (51)) + (2234817298.6145005 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.14913600146542363 = (2387076.4964690628 - 2743075.540344551) / 2387076.4964690628 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
786.43 ns 850.89 ns 1.08 0.13 False
765.87 ns 819.12 ns 1.07 0.18 False
523.53 ns 579.79 ns 1.11 0.17 False
498.27 ns 575.50 ns 1.15 0.16 False
768.93 ns 854.60 ns 1.11 0.11 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 850.893459779281 > 805.8658796392851. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.690647961612873 (T) = (0 -849.3929469971573) / Math.Sqrt((302.1978558313762 / (49)) + (249.5396002821591 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.10431528755654705 = (769.1580081957923 - 849.3929469971573) / 769.1580081957923 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 819.1239760654752 > 802.5629474185519. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -8.265741718077287 (T) = (0 -843.8302254901521) / Math.Sqrt((569.2038047766608 / (49)) + (307.2023397077713 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.10145189110721496 = (766.107201143308 - 843.8302254901521) / 766.107201143308 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 579.7905013798446 > 529.2555846712875. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.98338874233614 (T) = (0 -584.3837479539886) / Math.Sqrt((266.8222927801291 / (51)) + (19.81042321380662 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.1507117126002576 = (507.8454851506287 - 584.3837479539886) / 507.8454851506287 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 575.499801137092 > 522.4280891692491. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.056454512511142 (T) = (0 -573.7035717605372) / Math.Sqrt((229.83899816191717 / (50)) + (146.27667636496625 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.14109780804920544 = (502.7645901286303 - 573.7035717605372) / 502.7645901286303 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 854.6022169712102 > 820.0284693515058. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 11:21:57 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.628299791408654 (T) = (0 -847.7791475316618) / Math.Sqrt((326.7258026538941 / (42)) + (1172.4973363080926 / (11))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (11) - 2, .025) and -0.07642473501454103 = (787.5879473544516 - 847.7791475316618) / 787.5879473544516 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.ScopeValidation

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.56 μs 3.57 μs 1.40 0.09 True
2.47 μs 3.67 μs 1.49 0.06 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ScopeValidation*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.ScopeValidation.TransientWithScopeValidation #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.5721370103556045 > 2.6120338411876216. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.785264039048204 (T) = (0 -3621.334552260676) / Math.Sqrt((2982.00679607743 / (49)) + (6561.751511684511 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.46299851526088 = (2475.282452091159 - 3621.334552260676) / 2475.282452091159 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ScopeValidation.Transient #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.6721773304263565 > 2.538397354217031. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.97041877294565 (T) = (0 -3571.464970536285) / Math.Sqrt((3824.075075782129 / (50)) + (4607.074656322817 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.47827343088079644 = (2415.9704801082075 - 3571.464970536285) / 2415.9704801082075 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.48 μs 1.76 μs 1.19 0.12 False
1.59 μs 1.87 μs 1.17 0.18 False
2.32 μs 2.79 μs 1.21 0.12 False
1.21 μs 1.48 μs 1.22 0.16 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7629894654426332 > 1.5334554374938363. IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.869732564389988 (T) = (0 -1708.4049071552156) / Math.Sqrt((1682.9397781614334 / (51)) + (1692.6720703828535 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.19174492480392372 = (1433.5323537763732 - 1708.4049071552156) / 1433.5323537763732 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8696111285011061 > 1.6849958322701724. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.482050844976513 (T) = (0 -1885.5776661835157) / Math.Sqrt((1461.8879202561432 / (50)) + (720.5568242253956 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.17578538005376304 = (1603.675039825123 - 1885.5776661835157) / 1603.675039825123 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.7936287893727556 > 2.431156968575569. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.758170418713044 (T) = (0 -2827.0966293718484) / Math.Sqrt((3909.8027757387304 / (49)) + (5965.154599957528 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.21783974181810864 = (2321.402835114648 - 2827.0966293718484) / 2321.402835114648 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4796633165829147 > 1.2868926319132117. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -33.60179552569488 (T) = (0 -1465.8720589968648) / Math.Sqrt((1335.7772639851544 / (51)) + (91.34923773636048 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.1911946630385935 = (1230.5898477227915 - 1465.8720589968648) / 1230.5898477227915 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.21 μs 2.36 μs 1.07 0.02 False
1.44 μs 1.60 μs 1.11 0.03 False
1.86 μs 1.97 μs 1.06 0.01 False
2.95 μs 3.15 μs 1.07 0.03 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3612361376114377 > 2.337106216145719. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.76401089549868 (T) = (0 -2362.6533632701603) / Math.Sqrt((917.9466631473354 / (51)) + (32.77376372520845 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.06154265763469063 = (2225.679153143577 - 2362.6533632701603) / 2225.679153143577 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5954931427588475 > 1.5093338955328477. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.296257224427567 (T) = (0 -1553.558409715717) / Math.Sqrt((484.3969367690402 / (49)) + (1036.170461410555 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.08344444366559721 = (1433.9068503222852 - 1553.558409715717) / 1433.9068503222852 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9671978408183979 > 1.9537631531696555. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.10151077023108 (T) = (0 -1970.0606222557108) / Math.Sqrt((676.262738789881 / (50)) + (36.7288562860563 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.05918576868427504 = (1859.9764842978661 - 1970.0606222557108) / 1859.9764842978661 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.148498728527992 > 3.079961449913841. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.496460061592272 (T) = (0 -3205.413443014745) / Math.Sqrt((2346.560130883036 / (49)) + (2233.148495540168 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.08769291575675575 = (2946.9838376069574 - 3205.413443014745) / 2946.9838376069574 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
20.46 μs 22.44 μs 1.10 0.07 False
24.36 μs 29.18 μs 1.20 0.08 False
9.04 μs 11.72 μs 1.30 0.17 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<Location>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.443702651515153 > 21.446833560532074. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.064644992338955 (T) = (0 -22702.375343823922) / Math.Sqrt((340946.7200823265 / (49)) + (142637.84989576 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.11187024155529572 = (20418.1877483002 - 22702.375343823922) / 20418.1877483002 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<Location>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.181711646492865 > 25.707908662585808. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.10929687289938 (T) = (0 -29242.597617139985) / Math.Sqrt((616233.6229653957 / (50)) + (368811.842379073 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.16950435086699572 = (25004.265777601762 - 29242.597617139985) / 25004.265777601762 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<Location>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.721943197703103 > 9.439256304308122. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.69823803394302 (T) = (0 -11385.292470207693) / Math.Sqrt((75718.74419533323 / (51)) + (56979.6462115993 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.24217365384952852 = (9165.620631965889 - 11385.292470207693) / 9165.620631965889 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.85 ms 5.72 ms 1.18 0.06 False
641.75 μs 763.30 μs 1.19 0.18 False
1.89 ms 2.62 ms 1.38 0.11 False
595.32 μs 758.17 μs 1.27 0.08 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.722136507936509 > 5.075294564446079. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -29.496558286140353 (T) = (0 -5774966.449513591) / Math.Sqrt((24841224615.99735 / (50)) + (1936688668.0975869 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.19044260056921936 = (4851108.694154801 - 5774966.449513591) / 4851108.694154801 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 763.3047156593407 > 637.4059162504623. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -60.66450067383461 (T) = (0 -760870.4220352154) / Math.Sqrt((256032248.97203276 / (49)) + (4855158.874318186 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2536298465648892 = (606933.8761518007 - 760870.4220352154) / 606933.8761518007 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.6202028867781157 > 1.9947930587521854. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -41.57063444131317 (T) = (0 -2594659.4977098256) / Math.Sqrt((1489070693.8322172 / (51)) + (759601897.6266135 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.368435530093083 = (1896077.2653523057 - 2594659.4977098256) / 1896077.2653523057 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 758.173699490316 > 620.5252297832076. IsChangePoint: Marked as a change because one of 8/13/2023 2:50:54 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -35.1949412904858 (T) = (0 -749602.0252239524) / Math.Sqrt((219281696.21500066 / (50)) + (53461769.88322117 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2465970912777687 = (601318.6060426359 - 749602.0252239524) / 601318.6060426359 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<StructRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.02 μs 6.34 μs 1.26 0.12 True
6.44 μs 7.89 μs 1.23 0.12 False
5.54 μs 6.90 μs 1.25 0.12 True
5.95 μs 7.47 μs 1.26 0.07 True
5.24 μs 6.96 μs 1.33 0.12 True
6.36 μs 7.76 μs 1.22 0.09 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;StructRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.336314949270307 > 5.245754427992908. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.388151854338602 (T) = (0 -6455.824100448592) / Math.Sqrt((11925.388739778005 / (51)) + (8466.974273817346 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2709593341900615 = (5079.489112499941 - 6455.824100448592) / 5079.489112499941 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.8906534084719295 > 6.674361640026115. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.218229575445466 (T) = (0 -7937.338213010487) / Math.Sqrt((41092.181056410576 / (50)) + (7627.399601342729 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.218216045979089 = (6515.542328644335 - 7937.338213010487) / 6515.542328644335 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.901024153588269 > 5.821366009135901. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.71968045484871 (T) = (0 -7005.62292960716) / Math.Sqrt((24968.0782705336 / (51)) + (18452.46489373896 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.25167794085275735 = (5596.985215569342 - 7005.62292960716) / 5596.985215569342 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.465522166587188 > 6.211875001584773. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.220045378998028 (T) = (0 -7406.862681192988) / Math.Sqrt((11542.359386435495 / (48)) + (51230.567501321784 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.25609041257624743 = (5896.759187900716 - 7406.862681192988) / 5896.759187900716 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.955107329663405 > 5.558636442690243. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.585701592875267 (T) = (0 -6687.1140472809975) / Math.Sqrt((17827.242205427945 / (51)) + (33230.90045852127 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2546192575597651 = (5329.994743016648 - 6687.1140472809975) / 5329.994743016648 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.761772019262363 > 6.4598572477989356. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -21.782759075041152 (T) = (0 -7651.515005322159) / Math.Sqrt((25589.943647456956 / (51)) + (15963.516954408818 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.23581395070578778 = (6191.478095025784 - 7651.515005322159) / 6191.478095025784 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.33 ms 1.66 ms 1.24 0.06 False
817.19 μs 1.07 ms 1.31 0.06 True
592.75 μs 734.33 μs 1.24 0.13 True
151.63 μs 194.33 μs 1.28 0.14 True
23.78 μs 27.60 μs 1.16 0.41 False
135.96 μs 178.17 μs 1.31 0.13 True
93.14 μs 129.24 μs 1.39 0.17 True
25.08 μs 27.42 μs 1.09 0.34 False
18.17 μs 20.89 μs 1.15 0.28 False
72.46 μs 76.35 μs 1.05 0.25 False
21.55 μs 25.66 μs 1.19 0.17 False
1.25 ms 1.57 ms 1.25 0.11 False

graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CtorFromCollection<Int32>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6575740298076922 > 1.375260905208333. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.844825726217106 (T) = (0 -1644680.6094524574) / Math.Sqrt((1071095877.5518084 / (51)) + (1593739068.7509177 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2458038986271667 = (1320176.1619664533 - 1644680.6094524574) / 1320176.1619664533 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0732344505555556 > 861.4242651875. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.693542860379196 (T) = (0 -1083492.39245394) / Math.Sqrt((179270333.03335118 / (51)) + (722621237.1858835 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.32743107949899053 = (816232.5029054466 - 1083492.39245394) / 816232.5029054466 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 734.3337606292517 > 586.1714845312499. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.90930247443798 (T) = (0 -755372.2298613589) / Math.Sqrt((173695686.9568377 / (52)) + (257997750.12307253 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.3329783861527779 = (566680.0285048153 - 755372.2298613589) / 566680.0285048153 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 194.32694003527337 > 156.26944164791496. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.416811831331195 (T) = (0 -196537.5687224454) / Math.Sqrt((11384330.117133103 / (51)) + (17082511.179319963 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3379280367908102 = (146896.96554522144 - 196537.5687224454) / 146896.96554522144 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.HashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.60158096187944 > 24.527526202038338. IsChangePoint: Marked as a change because one of 8/22/2023 10:50:09 AM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -4.485327115502104 (T) = (0 -25548.808622496897) / Math.Sqrt((1810605.496226513 / (39)) + (1522694.8281811753 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.07429846990677641 = (23781.85331001535 - 25548.808622496897) / 23781.85331001535 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ImmutableList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 178.16548214285714 > 142.44414395952026. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.235530470160164 (T) = (0 -180422.45533319373) / Math.Sqrt((18855731.458246406 / (50)) + (11759566.681640811 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.3064207959336959 = (138104.3962977076 - 180422.45533319373) / 138104.3962977076 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.SortedDictionaryDeepCopy(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 129.23924129098359 > 100.48305446828606. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -47.786541015713794 (T) = (0 -128258.91419145248) / Math.Sqrt((9523339.406236034 / (51)) + (1095805.1999196692 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3384781364481951 = (95824.43724617138 - 128258.91419145248) / 95824.43724617138 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.LinkedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.423070737969486 > 25.875421710372855. IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.348006423965183 (T) = (0 -27591.986242979146) / Math.Sqrt((1158377.5360448654 / (40)) + (1589543.7767142141 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.09510778554138972 = (25195.680833680188 - 27591.986242979146) / 25195.680833680188 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ConcurrentStack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.891262197993452 > 19.24646073648848. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 8:25:02 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -4.260644148940328 (T) = (0 -20027.949604327176) / Math.Sqrt((2015292.2040811558 / (39)) + (1169836.3572793694 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.08487107180617919 = (18461.133423885163 - 20027.949604327176) / 18461.133423885163 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 76.34969366830066 > 75.20680372555566. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.719259287707349 (T) = (0 -77968.89388276958) / Math.Sqrt((5867956.491521967 / (39)) + (1262646.8399195778 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.08698763375590998 = (71729.32925958015 - 77968.89388276958) / 71729.32925958015 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ImmutableStack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.66194754644588 > 22.638184026064945. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.073793868076504 (T) = (0 -23970.66823498079) / Math.Sqrt((250754.16322947244 / (39)) + (1014829.6956507354 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.09039458385453839 = (21983.480649953908 - 23970.66823498079) / 21983.480649953908 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5732205366071428 > 1.3107010792067308. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.53101645687781 (T) = (0 -1550963.8395833333) / Math.Sqrt((1453649067.000542 / (51)) + (2434809096.1236496 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.21671771308180512 = (1274711.317923466 - 1550963.8395833333) / 1274711.317923466 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
480.49 ns 550.49 ns 1.15 0.12 False
562.37 ns 683.28 ns 1.22 0.16 False
422.83 ns 526.02 ns 1.24 0.09 True
538.44 ns 638.08 ns 1.19 0.08 False
1.74 μs 2.01 μs 1.16 0.28 False
740.98 ns 831.48 ns 1.12 0.17 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'
### Payloads [Baseline]() [Compare]() ### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 0) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 550.4886839837037 > 493.4083150535499. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -21.27600447000733 (T) = (0 -556.4788621288907) / Math.Sqrt((111.68418498870264 / (50)) + (52.309149941539715 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.17592253920820677 = (473.22748189144244 - 556.4788621288907) / 473.22748189144244 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "name5", hostPortCount: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 683.2834415823503 > 608.5332316704016. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.964194361941024 (T) = (0 -672.143376770291) / Math.Sqrt((203.06417671938982 / (49)) + (135.33295049033126 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.15901713038347343 = (579.9253170209012 - 672.143376770291) / 579.9253170209012 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 0) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 526.0206143961277 > 455.13816073038794. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -20.143977283460504 (T) = (0 -520.265640595885) / Math.Sqrt((120.55381396112794 / (51)) + (66.34986682440483 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2027240074466949 = (432.57275765233555 - 520.265640595885) / 432.57275765233555 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "notfound", hostPortCount: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 638.0766804004594 > 578.5687517002698. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -35.678422758532996 (T) = (0 -636.3185545150793) / Math.Sqrt((161.92416590199431 / (51)) + (15.591335734870443 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17524977322776203 = (541.4326120374086 - 636.3185545150793) / 541.4326120374086 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.0105194393186325 > 1.9546905005494466. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.978777472240219 (T) = (0 -1935.9735887438717) / Math.Sqrt((3360.395119284008 / (39)) + (7142.967660303488 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.08156942984452979 = (1789.9670010293821 - 1935.9735887438717) / 1789.9670010293821 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 0) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 831.4793406613146 > 778.2333696080393. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.453184503033338 (T) = (0 -821.881164696636) / Math.Sqrt((483.9521942858789 / (50)) + (54.279921716276284 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.12140204609590531 = (732.9049983081148 - 821.881164696636) / 732.9049983081148 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.47 μs 5.06 μs 1.13 0.07 False
10.65 μs 11.68 μs 1.10 0.09 False
36.52 μs 41.74 μs 1.14 0.10 False
5.54 μs 6.45 μs 1.17 0.14 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromStream<Location>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.062637275058654 > 4.686312308807639. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.053958034857796 (T) = (0 -5094.791734405136) / Math.Sqrt((13659.617055775221 / (49)) + (6088.820018937004 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.14337239228448628 = (4455.933839915109 - 5094.791734405136) / 4455.933839915109 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<Location>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.684642221391174 > 10.81188632969008. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.959059347490138 (T) = (0 -11640.307223718934) / Math.Sqrt((117001.32490437996 / (49)) + (21353.523958094425 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.11782803804366802 = (10413.32550943243 - 11640.307223718934) / 10413.32550943243 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<Location>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 41.74354686857261 > 39.285597228314145. IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.09508311434479 (T) = (0 -41937.517350160895) / Math.Sqrt((648714.3715268668 / (48)) + (101058.99918493505 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.09849614117241903 = (38177.20953065999 - 41937.517350160895) / 38177.20953065999 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<Location>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.453526267901527 > 5.495967038852023. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.66173608658774 (T) = (0 -6578.190495756338) / Math.Sqrt((17589.835663683945 / (51)) + (37417.35866950243 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.23390356917345417 = (5331.203069752705 - 6578.190495756338) / 5331.203069752705 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.24 μs 5.16 μs 1.22 0.21 False
16.80 μs 19.31 μs 1.15 0.10 False
10.57 μs 11.87 μs 1.12 0.09 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.162177787919018 > 4.486933287508903. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.739915911649625 (T) = (0 -5198.812163707522) / Math.Sqrt((14093.456506978144 / (50)) + (10432.972501387883 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.20934157361271752 = (4298.878230223153 - 5198.812163707522) / 4298.878230223153 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.312013462732917 > 17.71669208813318. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.836734759785903 (T) = (0 -18825.184651706666) / Math.Sqrt((107842.41037653945 / (41)) + (953974.5556624011 / (13))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (13) - 2, .025) and -0.11126852325203747 = (16940.2662433165 - 18825.184651706666) / 16940.2662433165 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<LoginViewModel>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.872293503835273 > 11.077040259278679. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -8.456698231057686 (T) = (0 -11812.851366137336) / Math.Sqrt((62768.30297020849 / (51)) + (88806.39025058957 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.12307274641811213 = (10518.33142938676 - 11812.851366137336) / 10518.33142938676 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.01 secs 9.99 secs 1.42 0.04 True
1.65 secs 1.86 secs 1.12 0.35 False
7.03 secs 10.22 secs 1.45 0.08 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'
### Payloads [Baseline]() [Compare]() ### LinqBenchmarks.Order00LinqQueryX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.989727866733334 > 7.462386329895. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -54.537536443534194 (T) = (0 -10108328395.305954) / Math.Sqrt((11511176557694410 / (49)) + (10840559983598168 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4140167961720248 = (7148662181.857285 - 10108328395.305954) / 7148662181.857285 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### LinqBenchmarks.Where01LinqMethodNestedX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8584279997142856 > 1.7371891126125. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.907874415867362 (T) = (0 -1837542738.743119) / Math.Sqrt((5276083966504676 / (39)) + (2511828298313143 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.07145447486235314 = (1714998426.7686064 - 1837542738.743119) / 1714998426.7686064 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### LinqBenchmarks.Order00LinqMethodX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.217728600133333 > 7.5425748474475. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -39.25327955505472 (T) = (0 -10031047383.333334) / Math.Sqrt((18801097395270710 / (51)) + (19632807514420372 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3971570821146843 = (7179613167.154203 - 10031047383.333334) / 7179613167.154203 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
46.04 μs 56.92 μs 1.24 0.04 True
46.08 μs 49.23 μs 1.07 0.06 False
47.02 μs 58.63 μs 1.25 0.08 True
46.01 μs 55.40 μs 1.20 0.04 False
45.19 μs 48.20 μs 1.07 0.11 False
46.06 μs 55.11 μs 1.20 0.03 True
44.74 μs 48.99 μs 1.10 0.09 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;TreeRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 56.91970808804291 > 49.711412735376605. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.166871389097533 (T) = (0 -57366.76126877651) / Math.Sqrt((1021243.7735932113 / (51)) + (3366044.963904725 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.22052292534499557 = (47001.78921470164 - 57366.76126877651) / 47001.78921470164 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeToString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 49.231276210937494 > 47.50427956445982. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 2:54:53 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.444593633150706 (T) = (0 -48142.343646129215) / Math.Sqrt((1286494.1611737008 / (38)) + (1860773.2500367046 / (13))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (13) - 2, .025) and -0.06958322825175815 = (45010.37635455283 - 48142.343646129215) / 45010.37635455283 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 58.63389762126865 > 49.71417815733159. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.989755457207156 (T) = (0 -58631.70702846856) / Math.Sqrt((966710.5616386901 / (49)) + (3151298.0047187433 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.22513773861212105 = (47857.23692985624 - 58631.70702846856) / 47857.23692985624 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.39855197990544 > 49.027446529907316. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -30.31368367615643 (T) = (0 -55391.88553745548) / Math.Sqrt((1022928.6172868243 / (50)) + (284279.92824142386 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.19841697226373012 = (46220.878725393784 - 55391.88553745548) / 46220.878725393784 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeToWriter(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 48.197617066563474 > 46.99983939297504. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 9:33:39 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -8.901809251223481 (T) = (0 -48220.5650208722) / Math.Sqrt((1247965.1122327228 / (41)) + (1652701.67318864 / (12))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (12) - 2, .025) and -0.08190302687314037 = (44570.13597626837 - 48220.5650208722) / 44570.13597626837 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.107365038145545 > 48.862995766912526. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.65217995039113 (T) = (0 -55304.055428933265) / Math.Sqrt((748684.7588839732 / (49)) + (428531.56142863235 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.17596242615890773 = (47028.76061233952 - 55304.055428933265) / 47028.76061233952 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 48.99318542779261 > 48.09923755121903. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.326066850807044 (T) = (0 -48206.0319255042) / Math.Sqrt((713114.8963565578 / (39)) + (2302760.3125525634 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.05943708434154038 = (45501.55232244408 - 48206.0319255042) / 45501.55232244408 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Perf_Frozen<NotKnownComparable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
140.94 μs 187.53 μs 1.33 0.20 True
131.44 μs 167.82 μs 1.28 0.09 True
27.43 μs 36.54 μs 1.33 0.04 True
17.95 μs 22.47 μs 1.25 0.15 False
1.12 μs 1.43 μs 1.27 0.16 True
1.49 μs 1.81 μs 1.21 0.02 True
16.47 μs 20.84 μs 1.27 0.12 True
157.60 μs 191.45 μs 1.21 0.09 True
17.89 μs 22.96 μs 1.28 0.16 False
1.58 μs 1.89 μs 1.20 0.03 True
226.84 μs 303.05 μs 1.34 0.03 True

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen&lt;NotKnownComparable&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Perf_Frozen<NotKnownComparable>.TryGetValue_True(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 187.52570180722893 > 151.12738484852161. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -24.699020046809515 (T) = (0 -186609.3252925352) / Math.Sqrt((11627572.76822654 / (51)) + (11316910.485397637 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.30110865426130984 = (143423.32185814306 - 186609.3252925352) / 143423.32185814306 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.Contains_True(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 167.8201233599291 > 137.9241995106826. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.88999375769099 (T) = (0 -168907.85644537094) / Math.Sqrt((9875879.474151442 / (49)) + (3874068.725150278 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2668321449326649 = (133330.88927448142 - 168907.85644537094) / 133330.88927448142 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenSet(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 36.54370563089623 > 28.835982681585563. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -50.12411765136666 (T) = (0 -36217.21230443293) / Math.Sqrt((349985.5324037129 / (50)) + (100383.73865391065 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.32968940025426613 = (27237.34753206831 - 36217.21230443293) / 27237.34753206831 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenDictionary(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.468287019230768 > 18.886983913356662. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -45.58447105804024 (T) = (0 -22330.10864010989) / Math.Sqrt((201987.86766657844 / (50)) + (14922.000847368636 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.24040433688222748 = (18002.281978662628 - 22330.10864010989) / 18002.281978662628 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenSet(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4254705586685854 > 1.1681490404030932. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.83876620694512 (T) = (0 -1413.7349329434633) / Math.Sqrt((1254.117315701998 / (50)) + (1493.4310644077611 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.26504321547156257 = (1117.5388442492645 - 1413.7349329434633) / 1117.5388442492645 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.Contains_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8074435411278462 > 1.5535341111917818. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.056386022521227 (T) = (0 -1776.8634892321331) / Math.Sqrt((637.1732885685561 / (51)) + (595.9710633672181 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.19745290921193534 = (1483.869198999665 - 1776.8634892321331) / 1483.869198999665 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.Contains_True(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.84470124177293 > 17.193976228079688. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -40.19736223907399 (T) = (0 -20952.253305873786) / Math.Sqrt((75733.59398801347 / (51)) + (42648.591081409504 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.268147109831158 = (16521.942244274313 - 20952.253305873786) / 16521.942244274313 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenDictionary(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 191.45355711382112 > 162.9357782453589. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.45350546971243 (T) = (0 -192981.89410477527) / Math.Sqrt((16874882.323199034 / (51)) + (3418567.2738430034 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.22414730030041638 = (157645.9745141911 - 192981.89410477527) / 157645.9745141911 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.TryGetValue_True(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.960618267350927 > 18.675792772639348. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.753922952685414 (T) = (0 -22499.279350447974) / Math.Sqrt((241690.79754653404 / (51)) + (211033.09583993946 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.256347919926652 = (17908.478211801015 - 22499.279350447974) / 17908.478211801015 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.TryGetValue_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.894349565276766 > 1.6660520283694755. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.643935502433052 (T) = (0 -1874.8028876635576) / Math.Sqrt((586.4826800767423 / (48)) + (669.434821022196 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.19310866707356814 = (1571.3597088034192 - 1874.8028876635576) / 1571.3597088034192 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<NotKnownComparable>.ToFrozenSet(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 303.04653039148354 > 239.0482241621377. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -56.20878782195327 (T) = (0 -301819.8879048235) / Math.Sqrt((16823197.84445803 / (49)) + (5161475.865782938 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.31239778647846306 = (229975.9196597643 - 301819.8879048235) / 229975.9196597643 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.55 μs 5.67 μs 1.25 0.04 True
5.12 μs 6.32 μs 1.23 0.06 True
6.21 μs 7.80 μs 1.26 0.07 False
4.68 μs 6.04 μs 1.29 0.06 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.6728911390790095 > 4.7412343558376095. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.611833417192312 (T) = (0 -5685.166159332055) / Math.Sqrt((10767.926414661657 / (51)) + (26400.162004391776 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.269221729712672 = (4479.253723948667 - 5685.166159332055) / 4479.253723948667 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.319788668274771 > 5.247581859530345. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -30.304422524940563 (T) = (0 -6228.030642317533) / Math.Sqrt((15241.044837807749 / (50)) + (5646.972011574127 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2525011734574279 = (4972.474896071802 - 6228.030642317533) / 4972.474896071802 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.797081569802758 > 6.556452128210475. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.473255572251066 (T) = (0 -7631.893800085111) / Math.Sqrt((23732.2997897881 / (49)) + (12225.811866310129 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2124444194284261 = (6294.633945927979 - 7631.893800085111) / 6294.633945927979 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.043091579595414 > 4.924109214755985. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.714316126673424 (T) = (0 -6056.896574623868) / Math.Sqrt((10671.399696021617 / (51)) + (21419.40015820182 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2790310368310573 = (4735.535260841291 - 6056.896574623868) / 4735.535260841291 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.24 ms 1.43 ms 1.15 0.11 False
1.57 ms 2.12 ms 1.35 0.12 True
2.42 ms 3.02 ms 1.25 0.06 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4282390466720776 > 1.3086348996875001. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.27111505720311 (T) = (0 -1419573.3800818971) / Math.Sqrt((1008470604.8357935 / (51)) + (238620970.81095603 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.13763971739569011 = (1247823.329631648 - 1419573.3800818971) / 1247823.329631648 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1217881378531076 > 1.6735595015865388. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -49.547685486031476 (T) = (0 -2121036.8990004347) / Math.Sqrt((2982548091.8623176 / (50)) + (177012326.14246836 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.3125384056786555 = (1615980.8275505207 - 2121036.8990004347) / 1615980.8275505207 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0184505031328315 > 2.529633327387921. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -31.18431097197464 (T) = (0 -2992260.5596147664) / Math.Sqrt((2356421967.1080465 / (49)) + (1055498078.1518055 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.22559947352504234 = (2441466.9100733963 - 2992260.5596147664) / 2441466.9100733963 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
974.76 ns 1.07 μs 1.10 0.10 False
977.00 ns 1.14 μs 1.17 0.09 False
2.54 μs 2.70 μs 1.06 0.05 False
1.53 μs 1.66 μs 1.09 0.24 False
6.22 μs 6.82 μs 1.10 0.09 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0745204572322669 > 1027.1456786563729. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.949878680947515 (T) = (0 -1078.366261377906) / Math.Sqrt((732.2343261040229 / (49)) + (199.43500041496839 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.10701595313044344 = (974.1198926072193 - 1078.366261377906) / 974.1198926072193 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1423843063033348 > 1025.4515247811275. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.321831933140008 (T) = (0 -1136.944731528778) / Math.Sqrt((615.5297468106642 / (49)) + (2757.1440917287 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.14156055023603187 = (995.956571286298 - 1136.944731528778) / 995.956571286298 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.70043805573012 > 2.6585310245714733. IsChangePoint: Marked as a change because one of 9/17/2023 1:05:21 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.385269607290406 (T) = (0 -2759.081142477862) / Math.Sqrt((2167.2190664756017 / (51)) + (3747.4974002295307 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.07807982410492277 = (2559.2549649731113 - 2759.081142477862) / 2559.2549649731113 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: HelloWorld) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6639536354299203 > 1.625413948345632. IsChangePoint: Marked as a change because one of 9/25/2023 11:21:57 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.672178529561361 (T) = (0 -1647.1350067157255) / Math.Sqrt((2497.8431633835 / (42)) + (2178.8957936884035 / (11))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (11) - 2, .025) and -0.06952884609805299 = (1540.056645246124 - 1647.1350067157255) / 1540.056645246124 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: BasicJson) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.815552178070928 > 6.558403144441262. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.838749783817117 (T) = (0 -6566.8796710268625) / Math.Sqrt((16487.391188068395 / (40)) + (21129.784520131936 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.0552132215671444 = (6223.272734655557 - 6566.8796710268625) / 6223.272734655557 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.57 ms 1.86 ms 1.18 0.08 False
1.57 ms 1.81 ms 1.15 0.05 False
1.62 ms 1.86 ms 1.15 0.04 False
1.48 ms 1.76 ms 1.19 0.08 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8560758159722222 > 1.6477200888194445. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.55455391799516 (T) = (0 -1862234.925173611) / Math.Sqrt((1552898006.0431278 / (50)) + (1032830432.3815972 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.1909382540545702 = (1563670.424418394 - 1862234.925173611) / 1563670.424418394 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8074640697916666 > 1.623785377447917. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -39.56256300834903 (T) = (0 -1805568.4214890255) / Math.Sqrt((868286514.186556 / (51)) + (119501338.35169764 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17655102860181376 = (1534628.2291170333 - 1805568.4214890255) / 1534628.2291170333 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8581884880952382 > 1.636573529621528. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -35.09464431057338 (T) = (0 -1839676.3203136923) / Math.Sqrt((767071777.3756245 / (49)) + (184491986.34413576 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.1763861944238485 = (1563837.0537106646 - 1839676.3203136923) / 1563837.0537106646 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7593194449786325 > 1.5609549995880683. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -21.061210735799825 (T) = (0 -1791303.400963367) / Math.Sqrt((1131505259.6319342 / (51)) + (604906296.286138 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.18319383114018678 = (1513956.0009683066 - 1791303.400963367) / 1513956.0009683066 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
483.28 μs 773.39 μs 1.60 0.19 True
1.36 ms 1.82 ms 1.33 0.08 True
1.32 ms 1.81 ms 1.37 0.06 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Sort<IntStruct>.Array_Comparison(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 773.3941334166667 > 508.27026587346154. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.709920711079594 (T) = (0 -812831.7475008334) / Math.Sqrt((138378782.349969 / (49)) + (1200194729.7141006 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.6683104499620695 = (487218.5195023587 - 812831.7475008334) / 487218.5195023587 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<IntStruct>.LinqQuery(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.81570602672 > 1.385719845972. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -31.145875115290973 (T) = (0 -1790623.0033433335) / Math.Sqrt((1242403595.7024255 / (51)) + (797833965.2331177 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3517581696959129 = (1324662.2387687482 - 1790623.0033433335) / 1324662.2387687482 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<IntStruct>.LinqOrderByExtension(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.81135576 > 1.3795573050490002. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -60.84864364166618 (T) = (0 -1791719.1833433332) / Math.Sqrt((304905560.67038685 / (50)) + (224344038.066995 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.3657587205791113 = (1311885.5888275825 - 1791719.1833433332) / 1311885.5888275825 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.40 ms 1.86 ms 1.32 0.03 False
532.57 μs 863.29 μs 1.62 0.10 False
1.42 ms 1.88 ms 1.32 0.04 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8589082132666668 > 1.4894973310699997. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -38.519011972393926 (T) = (0 -1881126.3332933334) / Math.Sqrt((624249600.743951 / (51)) + (378044811.1787607 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.317119526284035 = (1428212.3191966643 - 1881126.3332933334) / 1428212.3191966643 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 863.2941000166667 > 573.0621115160001. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.41004588406041 (T) = (0 -888691.9563539681) / Math.Sqrt((253099389.86813015 / (50)) + (500102648.9657317 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.6379416993969469 = (542566.2932210374 - 888691.9563539681) / 542566.2932210374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<BigStruct>.LinqQuery(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.877070213293333 > 1.4913045799499998. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.086730681392647 (T) = (0 -1921059.3607215383) / Math.Sqrt((407717700.4901247 / (48)) + (4880449509.601135 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.3459408035464758 = (1427298.5525512404 - 1921059.3607215383) / 1427298.5525512404 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.73 μs 1.99 μs 1.15 0.11 False
367.02 ns 407.44 ns 1.11 0.10 False
1.32 μs 1.41 μs 1.07 0.15 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments_DefineMessage #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9921020095618853 > 1.8172953536667964. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.271790373362695 (T) = (0 -2013.2291452367042) / Math.Sqrt((1687.8064615742883 / (48)) + (1927.5894977744101 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.17633465418866887 = (1711.4425202624425 - 2013.2291452367042) / 1711.4425202624425 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.FormattingOverhead.NoArguments #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 407.44404276286735 > 388.31142162979603. IsChangePoint: Marked as a change because one of 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.059948397173813 (T) = (0 -391.2503502551013) / Math.Sqrt((81.86360856630607 / (40)) + (226.60063682090615 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.05844978698848201 = (369.64469648417895 - 391.2503502551013) / 369.64469648417895 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4088286717070808 > 1.3884849273466155. IsChangePoint: Marked as a change because one of 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.688533101546755 (T) = (0 -1406.9258539350237) / Math.Sqrt((940.7208631565971 / (39)) + (2549.4942772645004 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.061642607995941276 = (1325.234917418087 - 1406.9258539350237) / 1325.234917418087 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.09 μs 2.91 μs 1.39 0.11 True
910.65 ns 1.27 μs 1.40 0.11 True
1.81 μs 2.61 μs 1.44 0.11 True
1.55 μs 2.09 μs 1.35 0.09 True
4.94 μs 6.09 μs 1.23 0.11 False
1.22 μs 1.69 μs 1.39 0.14 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_4Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9056073789571695 > 2.2064085438826297. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.643409775383013 (T) = (0 -2889.446269542152) / Math.Sqrt((4267.651277425106 / (51)) + (2524.5702223456396 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.36061557949741435 = (2123.631621658675 - 2889.446269542152) / 2123.631621658675 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_0Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2722343773975757 > 926.986109788165. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -39.506701022785805 (T) = (0 -1246.9935597690342) / Math.Sqrt((683.0839628809694 / (51)) + (305.67000555737667 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4290663428460831 = (872.5931906601071 - 1246.9935597690342) / 872.5931906601071 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_3Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.6115747709551655 > 1.9060114423584065. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.730110559081242 (T) = (0 -2456.321978536238) / Math.Sqrt((2147.251396102249 / (52)) + (11322.258226528478 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.3439351833600871 = (1827.7086640406108 - 2456.321978536238) / 1827.7086640406108 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_2Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.0899965197527544 > 1.6301415767350702. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -45.240940468279454 (T) = (0 -2093.5723195169526) / Math.Sqrt((1137.9228906342473 / (51)) + (500.0947558617145 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3555642154747936 = (1544.4287298360614 - 2093.5723195169526) / 1544.4287298360614 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_5Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.088678178712313 > 5.257858499738231. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.004412608845374 (T) = (0 -6037.170545433062) / Math.Sqrt((14467.912573049 / (49)) + (16471.899987358138 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.21375484744803847 = (4973.962046888152 - 6037.170545433062) / 4973.962046888152 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.GetService_1Injected #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6864262602992197 > 1.2958261793163262. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.623043990613425 (T) = (0 -1704.5621416864815) / Math.Sqrt((1510.9996908906605 / (49)) + (2603.3958514346673 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.36974218936874886 = (1244.4401252414045 - 1704.5621416864815) / 1244.4401252414045 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Threading.Channels.Tests.BoundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
578.15 ns 813.95 ns 1.41 0.26 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### System.Threading.Channels.Tests.BoundedChannelPerfTests.TryWriteThenTryRead #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 813.9472104638559 > 597.7637272704812. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -35.95077300618256 (T) = (0 -833.7352607856044) / Math.Sqrt((374.36836314064885 / (51)) + (176.8974034583329 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4485197988562041 = (575.577400767285 - 833.7352607856044) / 575.577400767285 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.36 μs 7.80 μs 1.22 0.13 False
3.51 μs 4.38 μs 1.25 0.03 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<Location>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.795293211175767 > 6.753395267665461. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.873076444092685 (T) = (0 -7529.907557888454) / Math.Sqrt((42664.09114489326 / (49)) + (35039.102675109265 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.18306533854477672 = (6364.743613527362 - 7529.907557888454) / 6364.743613527362 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<Location>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.382192952307168 > 3.724286215826771. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.17556198283109 (T) = (0 -4286.0677871922835) / Math.Sqrt((6450.542925774637 / (49)) + (5495.168222080848 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.21005047617161338 = (3542.0570229042405 - 4286.0677871922835) / 3542.0570229042405 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.74 μs 4.25 μs 1.14 0.06 False
4.33 μs 5.32 μs 1.23 0.06 False
2.97 μs 3.73 μs 1.25 0.05 True
3.26 μs 4.09 μs 1.26 0.09 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.248775824644318 > 3.727445103626883. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.31642564829539 (T) = (0 -4268.289769002717) / Math.Sqrt((9395.675022935498 / (51)) + (4575.614620441105 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.19750109357626222 = (3564.3305813239267 - 4268.289769002717) / 3564.3305813239267 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.324377477362762 > 4.5258938376182085. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.943489523721219 (T) = (0 -5314.581141577098) / Math.Sqrt((10964.876359415683 / (49)) + (15590.851279312848 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2202764789848623 = (4355.227059689172 - 5314.581141577098) / 4355.227059689172 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.728687965141146 > 3.1171112482342473. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.282440419059807 (T) = (0 -3738.086599317912) / Math.Sqrt((4745.814373789386 / (49)) + (13938.219217119813 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.24476518886649382 = (3003.0455806061564 - 3738.086599317912) / 3003.0455806061564 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.094908498521996 > 3.4512919692438184. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -20.13754476615844 (T) = (0 -4052.425622330989) / Math.Sqrt((8581.585713826615 / (50)) + (4631.416382947271 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.22128398357832466 = (3318.1681548443025 - 4052.425622330989) / 3318.1681548443025 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
62.10 μs 78.28 μs 1.26 0.03 True
68.39 μs 84.89 μs 1.24 0.08 True
66.56 μs 82.74 μs 1.24 0.05 True
50.17 μs 67.11 μs 1.34 0.13 True
54.71 μs 69.41 μs 1.27 0.08 True
56.17 μs 70.98 μs 1.26 0.11 True
64.66 μs 78.38 μs 1.21 0.09 True

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.27548130234507 > 65.76856722473141. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -49.675051714096824 (T) = (0 -78523.00741344581) / Math.Sqrt((1954650.1120229121 / (50)) + (229725.8968117725 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.24462346613851788 = (63089.76935576012 - 78523.00741344581) / 63089.76935576012 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 84.88510588951083 > 71.38636576615758. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.97853244851889 (T) = (0 -83528.48508671646) / Math.Sqrt((1802279.392041349 / (49)) + (1028205.0987115636 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.22184918319591734 = (68362.35292823623 - 83528.48508671646) / 68362.35292823623 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.73748579977205 > 71.31902199732464. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -40.80835576363664 (T) = (0 -82289.92447429968) / Math.Sqrt((2137429.3249177486 / (51)) + (395262.4941146444 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2285507881789993 = (66981.29639090678 - 82289.92447429968) / 66981.29639090678 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 67.11237686653327 > 52.83981798333977. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.70955163682225 (T) = (0 -68673.89402323433) / Math.Sqrt((2430805.4014970516 / (49)) + (8702812.438107016 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.3411246802916597 = (51206.196584421625 - 68673.89402323433) / 51206.196584421625 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 69.40755883112094 > 59.23357230724887. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.250660132043564 (T) = (0 -70752.08747541308) / Math.Sqrt((1615169.2621884958 / (47)) + (836233.0857821776 / (4))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (4) - 2, .025) and -0.2453331320387827 = (56813.78392268591 - 70752.08747541308) / 56813.78392268591 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 70.97659782697326 > 60.705017556179776. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.673441898860325 (T) = (0 -71559.94397819055) / Math.Sqrt((1795856.812720555 / (49)) + (961545.63176216 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.24407560530641445 = (57520.57485329874 - 71559.94397819055) / 57520.57485329874 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.3795190920398 > 67.39235381287568. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.216988451134 (T) = (0 -77948.19264686778) / Math.Sqrt((2460380.7489516786 / (49)) + (712108.661901841 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.24604007195993752 = (62556.7302376243 - 77948.19264686778) / 62556.7302376243 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
307.99 ns 394.69 ns 1.28 0.20 False
164.49 ns 234.43 ns 1.43 0.22 True
169.70 ns 247.11 ns 1.46 0.32 True
199.97 ns 270.23 ns 1.35 0.22 True
811.80 ns 927.98 ns 1.14 0.08 False
4.07 μs 4.91 μs 1.21 0.27 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'
### Payloads [Baseline]() [Compare]() ### System.Reflection.Invoke.Method0_NoParms #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 394.68895854166664 > 323.71749863238125. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.243476247148937 (T) = (0 -391.9376125835239) / Math.Sqrt((71.21205531316441 / (51)) + (45.60596392926279 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2547474143883566 = (312.363753922999 - 391.9376125835239) / 312.363753922999 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Invoke.Ctor0_NoParams #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 234.42503904584223 > 167.48086040880432. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -34.129831601630904 (T) = (0 -235.98665665603335) / Math.Sqrt((18.434074591244297 / (50)) + (18.685053561859853 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4807861283163441 = (159.36579371144603 - 235.98665665603335) / 159.36579371144603 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Invoke.Property_Get_class #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 247.10651067708335 > 178.42019455319817. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.885693316541452 (T) = (0 -242.19571457200755) / Math.Sqrt((26.738379242722434 / (49)) + (25.380597892273666 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4112347915585935 = (171.61971630852594 - 242.19571457200755) / 171.61971630852594 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Invoke.Property_Get_int #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 270.2292962592365 > 209.32416971205876. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.414473145962813 (T) = (0 -276.5579006643851) / Math.Sqrt((39.66107824305392 / (51)) + (42.458091761792055 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.37656842342675606 = (200.9038533485583 - 276.5579006643851) / 200.9038533485583 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Invoke.StaticMethod4_ByRefParams_int_string_struct_class #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 927.9831969537817 > 850.0272501328126. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -8.946964031485004 (T) = (0 -925.7561827667873) / Math.Sqrt((213.28207086355633 / (51)) + (606.3859092103288 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.13716966044421092 = (814.0880072417343 - 925.7561827667873) / 814.0880072417343 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Invoke.Ctor4_ActivatorCreateInstance #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.909723602083333 > 4.36762046984375. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.171227403115338 (T) = (0 -4888.4478344866075) / Math.Sqrt((27284.70352179208 / (49)) + (5317.710988095822 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.14403816946807488 = (4272.976168932817 - 4888.4478344866075) / 4272.976168932817 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.40 μs 3.09 μs 1.29 0.09 True
2.75 μs 3.29 μs 1.20 0.06 True
3.68 μs 4.47 μs 1.22 0.13 False
3.00 μs 3.94 μs 1.32 0.04 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0921917176904334 > 2.5144395759214393. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.08972432584993 (T) = (0 -3061.1546916830534) / Math.Sqrt((3693.6054543107766 / (52)) + (1984.9388272780498 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.27967619973249563 = (2392.1322380794136 - 3061.1546916830534) / 2392.1322380794136 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2887926167219783 > 2.7967871370184216. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.82887728440361 (T) = (0 -3361.598340702436) / Math.Sqrt((4194.018251397944 / (48)) + (5970.832375167887 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.24843678105202596 = (2692.6460287958694 - 3361.598340702436) / 2692.6460287958694 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.474734952655414 > 3.8021180278362614. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.302551254683225 (T) = (0 -4494.436028954796) / Math.Sqrt((10424.439877489742 / (51)) + (10390.006856843189 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.2183975808073977 = (3688.8090552317576 - 4494.436028954796) / 3688.8090552317576 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.943661716224264 > 3.0483833522213915. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -9.580587012964848 (T) = (0 -3701.906023287948) / Math.Sqrt((5299.796848232939 / (50)) + (28148.139884363787 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.27996693724159233 = (2892.188786739901 - 3701.906023287948) / 2892.188786739901 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.09 μs 2.59 μs 1.23 0.16 False
2.51 μs 3.15 μs 1.26 0.10 False
1.64 μs 2.16 μs 1.32 0.19 False
1.93 μs 2.46 μs 1.28 0.12 True
2.80 μs 3.39 μs 1.21 0.14 False
2.29 μs 2.97 μs 1.30 0.10 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.5861451145412406 > 2.181477995728843. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -30.02265931401307 (T) = (0 -2613.322315418126) / Math.Sqrt((5174.571255480067 / (48)) + (896.0772031048158 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.26465600181740506 = (2066.4293781570536 - 2613.322315418126) / 2066.4293781570536 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.152799716484407 > 2.672274532125848. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.6604862570426 (T) = (0 -3068.9039675156887) / Math.Sqrt((7875.802973165859 / (50)) + (3590.8128368820053 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.19870325707032754 = (2560.1865594460774 - 3068.9039675156887) / 2560.1865594460774 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.16260035773826 > 1.7507870811299875. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.01764609750642 (T) = (0 -2167.02188040557) / Math.Sqrt((4443.388751310821 / (51)) + (440.5553466110435 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2618540261823942 = (1717.3316686730122 - 2167.02188040557) / 1717.3316686730122 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.464019559474733 > 2.0219334510104225. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.72064608762096 (T) = (0 -2486.2672669576564) / Math.Sqrt((4542.453554770665 / (51)) + (2932.5445940129975 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2943945445738267 = (1920.7955390265092 - 2486.2672669576564) / 1920.7955390265092 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.3882350662696084 > 2.824010632056791. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.036231744765175 (T) = (0 -3409.4660353340178) / Math.Sqrt((5718.967196043675 / (51)) + (4857.375788245274 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2386479929943872 = (2752.5705887527865 - 3409.4660353340178) / 2752.5705887527865 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.971730186463653 > 2.3697792146011865. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -25.99749005372781 (T) = (0 -2905.032944674799) / Math.Sqrt((4542.996568152694 / (51)) + (2054.429404384109 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2815548446413013 = (2266.8034511530395 - 2905.032944674799) / 2266.8034511530395 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.49 ms 2.99 ms 1.20 0.04 False
537.68 μs 748.87 μs 1.39 0.12 True
679.34 μs 738.21 μs 1.09 0.28 False
2.40 ms 2.84 ms 1.18 0.04 False
497.89 μs 542.46 μs 1.09 0.16 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CreateAddAndClear<Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9858941798941805 > 2.621881162832643. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.446913649153696 (T) = (0 -2984784.4938051743) / Math.Sqrt((2067873083.9813106 / (49)) + (3235468399.4946785 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.19154150146802476 = (2504977.3676601322 - 2984784.4938051743) / 2504977.3676601322 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 748.8672352272727 > 567.461295093598. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.332624268407766 (T) = (0 -735658.8536756213) / Math.Sqrt((204461049.19016764 / (50)) + (412459290.46724373 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.3477456365624889 = (545843.9884487151 - 735658.8536756213) / 545843.9884487151 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<Int32>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 738.2073871212119 > 725.3015761820651. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.210970785986003 (T) = (0 -754927.91241416) / Math.Sqrt((977373101.2897781 / (41)) + (532357665.06628203 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.08123173721422085 = (698211.0184438552 - 754927.91241416) / 698211.0184438552 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<Int32>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.8350835233716474 > 2.5207850010679613. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.727808816795875 (T) = (0 -2827614.8278566636) / Math.Sqrt((1990782505.3960922 / (51)) + (1764707623.1097908 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.1697803744856729 = (2417218.556175474 - 2827614.8278566636) / 2417218.556175474 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<Int32>.ImmutableList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 542.4551556697613 > 517.7379225640792. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.14406746274229 (T) = (0 -539943.1514444283) / Math.Sqrt((793818965.5356381 / (41)) + (133952405.78559688 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.06519387837451289 = (506896.5963908675 - 539943.1514444283) / 506896.5963908675 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
127.16 μs 167.89 μs 1.32 0.14 True
103.70 μs 147.11 μs 1.42 0.09 True
103.66 μs 136.19 μs 1.31 0.13 True
120.60 μs 163.72 μs 1.36 0.09 True
125.94 μs 171.18 μs 1.36 0.09 True
101.24 μs 134.06 μs 1.32 0.10 True
105.68 μs 145.50 μs 1.38 0.13 False
120.86 μs 169.92 μs 1.41 0.08 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 167.88872549019612 > 133.2248610051186. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -39.06524888261003 (T) = (0 -168605.2277874075) / Math.Sqrt((15023339.842710463 / (49)) + (3444916.6611710875 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.33401882883954753 = (126388.94155195385 - 168605.2277874075) / 126388.94155195385 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 147.1094838888889 > 114.076444406596. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.317425232610375 (T) = (0 -145050.73857658915) / Math.Sqrt((7384042.832407976 / (50)) + (10293963.144434439 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.34013430589813565 = (108235.971527778 - 145050.73857658915) / 108235.971527778 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 136.18680665024633 > 107.74602593708882. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.0233541380518 (T) = (0 -138163.63674553973) / Math.Sqrt((7235496.11851829 / (48)) + (6879908.672579545 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.36521723821215984 = (101202.67520682198 - 138163.63674553973) / 101202.67520682198 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 163.71509384498484 > 126.95054933444653. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -36.39701567626639 (T) = (0 -164533.40317094116) / Math.Sqrt((10493364.403887518 / (51)) + (4535512.525620164 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3441544210821161 = (122406.62277365646 - 164533.40317094116) / 122406.62277365646 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 171.17541652777783 > 131.78596582721818. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -47.35965263027709 (T) = (0 -169794.68633572236) / Math.Sqrt((11307635.511597993 / (50)) + (2373398.0128889 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.33779066126738466 = (126921.7159692711 - 169794.68633572236) / 126921.7159692711 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 134.05901623563219 > 104.19077583677903. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -42.9112931292594 (T) = (0 -135785.97242726292) / Math.Sqrt((10907547.688068427 / (51)) + (1859286.652608509 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.35198695517675105 = (100434.38060355475 - 135785.97242726292) / 100434.38060355475 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 145.49582781208278 > 110.40797641192921. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -47.51667943413278 (T) = (0 -144145.42728507397) / Math.Sqrt((8465682.630414145 / (50)) + (1378907.4259793875 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.3539652265496512 = (106461.6907868483 - 144145.42728507397) / 106461.6907868483 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 169.91611171296296 > 127.8736565048077. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.73464802056134 (T) = (0 -167577.94976808914) / Math.Sqrt((9398800.047543913 / (51)) + (9515135.100011064 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3783924528605796 = (121574.91824647937 - 167577.94976808914) / 121574.91824647937 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
477.69 μs 672.34 μs 1.41 0.16 True
128.68 μs 142.74 μs 1.11 0.20 False
161.13 μs 240.61 μs 1.49 0.14 True
104.24 μs 109.65 μs 1.05 0.12 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.TryGetValueFalse<Int32, Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 672.3427789208075 > 489.88467719301474. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -29.71309396568075 (T) = (0 -661217.1251825256) / Math.Sqrt((117704235.83952145 / (50)) + (161905667.4485155 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.41661132698735154 = (466759.7333057534 - 661217.1251825256) / 466759.7333057534 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 142.7377533994102 > 133.97938325094265. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.657029237517161 (T) = (0 -140575.55968502726) / Math.Sqrt((16150604.23155342 / (41)) + (19186023.279671375 / (14))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (14) - 2, .025) and -0.07799355198805741 = (130404.82424572483 - 140575.55968502726) / 130404.82424572483 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 240.6146975961539 > 170.22095519750792. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -71.21835778150226 (T) = (0 -239283.86656598613) / Math.Sqrt((14285548.535001978 / (49)) + (3067334.672049688 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.44132765224294424 = (166016.2879645172 - 239283.86656598613) / 166016.2879645172 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 109.6532745091447 > 109.13809776667833. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.176330785889414 (T) = (0 -113155.34320930377) / Math.Sqrt((7536030.9707502555 / (40)) + (6597765.562727621 / (14))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (14) - 2, .025) and -0.07879885530227632 = (104890.12168778949 - 113155.34320930377) / 104890.12168778949 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
90.51 μs 106.49 μs 1.18 0.04 False
97.52 μs 124.74 μs 1.28 0.04 False
109.43 μs 128.78 μs 1.18 0.07 False
108.04 μs 127.53 μs 1.18 0.06 False
90.83 μs 104.93 μs 1.16 0.02 False
124.75 μs 145.68 μs 1.17 0.04 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 106.4899471153846 > 97.40781245710788. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -20.91557181856686 (T) = (0 -107761.29663729129) / Math.Sqrt((4401596.352107561 / (49)) + (1743589.4285413392 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.16377851846589728 = (92596.0523651383 - 107761.29663729129) / 92596.0523651383 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 124.74173221784777 > 103.97118094634433. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.438020012334178 (T) = (0 -122304.62914001662) / Math.Sqrt((5870009.959744048 / (50)) + (3405885.872191407 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2324835872598023 = (99234.28628525446 - 122304.62914001662) / 99234.28628525446 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 128.77930447658403 > 116.12188473779324. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.930600250340774 (T) = (0 -130053.34781786126) / Math.Sqrt((9163413.58467756 / (49)) + (2287479.5580802877 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.15406958536927653 = (112691.07986781151 - 130053.34781786126) / 112691.07986781151 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 127.52534108606558 > 107.3762216732166. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.731538127994444 (T) = (0 -126472.49831172849) / Math.Sqrt((5014837.049740893 / (49)) + (5238046.0218339255 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.22756431806318989 = (103027.18680457622 - 126472.49831172849) / 103027.18680457622 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 104.92762359344424 > 93.87629148103935. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -30.910472790657433 (T) = (0 -103820.8503510304) / Math.Sqrt((3567546.172518368 / (51)) + (566175.572149329 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.15864327352916666 = (89605.5349588295 - 103820.8503510304) / 89605.5349588295 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 145.67823216510902 > 129.71911742861172. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.886076893777968 (T) = (0 -146461.96220861364) / Math.Sqrt((7173543.363626883 / (48)) + (1941379.0137529105 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.18643320636846064 = (123447.28841239813 - 146461.96220861364) / 123447.28841239813 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.03 μs 2.42 μs 1.19 0.09 True
2.26 μs 2.96 μs 1.31 0.13 True
15.90 μs 16.76 μs 1.05 0.03 False
4.81 μs 5.80 μs 1.21 0.15 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4186503322742694 > 2.1138731043666485. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -8.392358568370218 (T) = (0 -2503.702281074288) / Math.Sqrt((1753.4981490799553 / (51)) + (14046.44404730437 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.24938231873597272 = (2003.9520677764497 - 2503.702281074288) / 2003.9520677764497 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9623363091730504 > 2.402294092927051. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -39.85784013207219 (T) = (0 -2986.2747248284904) / Math.Sqrt((4095.2846237446265 / (50)) + (894.7698408364362 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.3043319029659087 = (2289.5052386881184 - 2986.2747248284904) / 2289.5052386881184 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.76129792787944 > 16.562799414157215. IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/25/2023 7:14:09 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.7378045545839615 (T) = (0 -16877.629897355506) / Math.Sqrt((93408.53374891964 / (44)) + (249210.22203319482 / (9))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (9) - 2, .025) and -0.08596562414511184 = (15541.587617602376 - 16877.629897355506) / 15541.587617602376 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.799655861111112 > 5.193780729546862. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 10:28:34 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.789273168571568 (T) = (0 -5637.140646116426) / Math.Sqrt((24705.889269713618 / (45)) + (48355.72932551139 / (8))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (8) - 2, .025) and -0.1263807163554262 = (5004.649462000948 - 5637.140646116426) / 5004.649462000948 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
871.17 ns 1.18 μs 1.35 0.13 True
922.64 ns 1.20 μs 1.30 0.08 True
6.39 μs 7.17 μs 1.12 0.07 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: False) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1778345048269878 > 934.7803172323318. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -21.268986510359245 (T) = (0 -1185.4880109126943) / Math.Sqrt((540.7913171318789 / (49)) + (733.0570884392902 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.3334837584582256 = (889.0157104600629 - 1185.4880109126943) / 889.0157104600629 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1976037476358226 > 955.3878390227565. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.508210506118473 (T) = (0 -1177.480866155432) / Math.Sqrt((592.6263083719042 / (51)) + (377.83111546825535 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3168534001567813 = (894.1624527189163 - 1177.480866155432) / 894.1624527189163 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: True, Json: False) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.173946441779374 > 6.611144546691158. IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/26/2023 6:21:18 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.355130199478978 (T) = (0 -6954.953068467188) / Math.Sqrt((13916.869976165463 / (49)) + (15622.723101545043 / (6))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (6) - 2, .025) and -0.11506346151198914 = (6237.271068891901 - 6954.953068467188) / 6237.271068891901 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
92.83 μs 112.16 μs 1.21 0.03 False
84.34 μs 101.66 μs 1.21 0.05 False
96.41 μs 110.39 μs 1.14 0.04 False
90.89 μs 107.40 μs 1.18 0.06 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 112.16001139088729 > 97.71059737893884. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -31.13759349591331 (T) = (0 -110921.3407012225) / Math.Sqrt((2580608.4283008464 / (50)) + (922332.1040757544 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.17525805034606698 = (94380.41344925105 - 110921.3407012225) / 94380.41344925105 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 101.66271709057071 > 88.49440290944675. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.825556969155823 (T) = (0 -100576.6244693736) / Math.Sqrt((5188846.566888608 / (52)) + (936603.7460726317 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.19029810511174663 = (84497.00460535585 - 100576.6244693736) / 84497.00460535585 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 110.38803926566192 > 100.0133249258321. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -38.49272219860525 (T) = (0 -110852.38812229387) / Math.Sqrt((2430504.3554841243 / (51)) + (347760.67747655464 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.16338771346874703 = (95284.13171201311 - 110852.38812229387) / 95284.13171201311 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 107.40403263546798 > 94.4524114104118. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -39.686034612319 (T) = (0 -106439.24300170982) / Math.Sqrt((2603652.4924170077 / (48)) + (513968.35288296436 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.18960479926307613 = (89474.45661588259 - 106439.24300170982) / 89474.45661588259 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.86 μs 9.52 μs 1.21 0.15 False
3.33 μs 3.70 μs 1.11 0.12 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.524131639752184 > 8.372461632325914. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.26436490104848 (T) = (0 -9437.644741186403) / Math.Sqrt((47449.31472693986 / (51)) + (71816.07620271172 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17572130289433102 = (8027.110436761916 - 9437.644741186403) / 8027.110436761916 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.701912081337188 > 3.501207233619583. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/25/2023 2:54:53 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -5.923982813933151 (T) = (0 -3568.0526778245517) / Math.Sqrt((5261.411560720138 / (40)) + (16708.701127673692 / (13))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (13) - 2, .025) and -0.06666009337286663 = (3345.0699993304115 - 3568.0526778245517) / 3345.0699993304115 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
652.91 ns 910.61 ns 1.39 0.19 True
142.36 μs 193.27 μs 1.36 0.18 True
137.89 μs 205.99 μs 1.49 0.10 False
787.08 ns 1.10 μs 1.39 0.12 False
639.91 ns 910.10 ns 1.42 0.16 True
139.64 μs 200.32 μs 1.43 0.12 True
147.36 μs 213.67 μs 1.45 0.18 True
777.63 ns 1.13 μs 1.45 0.16 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'
### Payloads [Baseline]() [Compare]() ### System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsSmall #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 910.6079902350924 > 679.8648226200595. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.4585995410601 (T) = (0 -907.7272552445993) / Math.Sqrt((321.6235186417771 / (51)) + (226.88123850987787 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3964162118761967 = (650.0406164899756 - 907.7272552445993) / 650.0406164899756 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsLarge #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 193.26900763888892 > 144.6944850566197. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.124014353526437 (T) = (0 -198878.5957297591) / Math.Sqrt((15080335.48481601 / (50)) + (41570733.6702004 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.42453772527729317 = (139609.21652043046 - 198878.5957297591) / 139609.21652043046 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityEventsLarge #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 205.9937734375 > 144.78733010597827. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -63.67711374510129 (T) = (0 -205091.85867384102) / Math.Sqrt((15602633.291357584 / (51)) + (2211781.9837025376 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.46437672090465365 = (140054.02827432318 - 205091.85867384102) / 140054.02827432318 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0973480264249318 > 841.1736183916825. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.888284537082123 (T) = (0 -1137.822990220219) / Math.Sqrt((640.8125825429194 / (50)) + (1805.9280999984203 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.43399645642572693 = (793.462902311678 - 1137.822990220219) / 793.462902311678 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsSmall #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 910.1027216318801 > 678.886861070485. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.55878680743987 (T) = (0 -908.0111764257622) / Math.Sqrt((327.5899387261387 / (49)) + (302.40973117847324 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.3992076190315625 = (648.9467067469419 - 908.0111764257622) / 648.9467067469419 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 200.32174155982906 > 147.47807492617545. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -87.99599321470578 (T) = (0 -201556.12952450576) / Math.Sqrt((10242305.157031475 / (51)) + (1202154.7773531806 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.44745647410214573 = (139248.4908049001 - 201556.12952450576) / 139248.4908049001 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityLinksLarge #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 213.66560677592955 > 147.26657388294245. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -24.18945631294752 (T) = (0 -207766.2716340576) / Math.Sqrt((11545420.861988332 / (51)) + (29801848.180868775 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4761970796144479 = (140744.2640980714 - 207766.2716340576) / 140744.2640980714 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1263267439363838 > 815.4613498645917. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.294530984306387 (T) = (0 -1100.7199742629873) / Math.Sqrt((373.38931894258866 / (48)) + (462.4257115016792 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.3996462912379219 = (786.427243192601 - 1100.7199742629873) / 786.427243192601 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
559.59 ns 741.37 ns 1.32 0.13 False
24.28 μs 35.01 μs 1.44 0.09 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Scoped #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 741.3679180043454 > 604.4178824203908. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.436054196360484 (T) = (0 -712.098224766534) / Math.Sqrt((249.48154276768398 / (50)) + (418.9911381074598 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.246347637890632 = (571.347995629628 - 712.098224766534) / 571.347995629628 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Transient #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.01441692849189 > 26.279352921803387. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -52.078305930435846 (T) = (0 -35427.111365804754) / Math.Sqrt((304825.9544440407 / (49)) + (134240.08453555146 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.4148203153179752 = (25040.007541765226 - 35427.111365804754) / 25040.007541765226 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.21 ms 2.85 ms 1.29 0.11 False
1.25 ms 1.48 ms 1.18 0.16 False
3.00 ms 3.62 ms 1.21 0.05 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.8492890340531556 > 2.3106089214161036. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -25.458059984768315 (T) = (0 -2874445.217989352) / Math.Sqrt((4113264735.3243713 / (51)) + (2155020723.910799 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2827490684390628 = (2240847.636309083 - 2874445.217989352) / 2240847.636309083 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4792560872564935 > 1.3336955246394229. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.14105103313694 (T) = (0 -1494342.6379563147) / Math.Sqrt((1255868961.6801226 / (50)) + (131254488.43350573 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.1604151497699361 = (1287765.5365431786 - 1494342.6379563147) / 1287765.5365431786 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.6155331242236026 > 3.145954917092743. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.856769530102778 (T) = (0 -3689769.897285629) / Math.Sqrt((6950600530.656714 / (49)) + (4864920139.1497345 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2170589789096713 = (3031710.016708631 - 3689769.897285629) / 3031710.016708631 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.73 μs 10.14 μs 1.31 0.09 True
7.66 μs 9.24 μs 1.21 0.11 False
8.13 μs 10.06 μs 1.24 0.10 False
9.16 μs 11.08 μs 1.21 0.04 True
7.71 μs 9.96 μs 1.29 0.12 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.136859519381817 > 8.118086745689213. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.611389471807687 (T) = (0 -10030.265586043675) / Math.Sqrt((42261.529143106716 / (49)) + (20286.565497146876 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2568776663045377 = (7980.303775732277 - 10030.265586043675) / 7980.303775732277 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.242707661118677 > 7.952731215010693. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.034186076089659 (T) = (0 -9475.649860317564) / Math.Sqrt((27714.612281676396 / (47)) + (64471.36289238339 / (4))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (4) - 2, .025) and -0.2367675740091285 = (7661.625401126198 - 9475.649860317564) / 7661.625401126198 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.063711311154842 > 8.54493717475192. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.721283139419691 (T) = (0 -9883.176908011908) / Math.Sqrt((38132.33673629173 / (50)) + (49434.615462057096 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.20572427680803726 = (8196.87974946979 - 9883.176908011908) / 8196.87974946979 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.078164237841044 > 9.536948692717884. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.13572283793201 (T) = (0 -11175.15457248608) / Math.Sqrt((34678.70683160066 / (50)) + (36978.007393337 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.24605241865065627 = (8968.446596000831 - 11175.15457248608) / 8968.446596000831 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.962163229740765 > 7.746835081562373. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.602687247225779 (T) = (0 -9458.59513503526) / Math.Sqrt((35155.2300012208 / (51)) + (118281.35320142866 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.27126998229795835 = (7440.272535923348 - 9458.59513503526) / 7440.272535923348 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.30 μs 3.61 μs 1.10 0.01 False
4.61 μs 4.93 μs 1.07 0.00 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;BinaryData&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.612802603232715 > 3.4794996215805103. IsChangePoint: Marked as a change because one of 9/15/2023 9:01:53 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.133288739633707 (T) = (0 -3589.721588688468) / Math.Sqrt((2909.631350833954 / (51)) + (958.8291359834612 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.07831015969161348 = (3329.0251013818643 - 3589.721588688468) / 3329.0251013818643 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.92863937024514 > 4.855924514575633. IsChangePoint: Marked as a change because one of 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.472580101044249 (T) = (0 -4829.194027149335) / Math.Sqrt((2334.9496989334953 / (39)) + (13185.839006063461 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.05149629805536213 = (4592.687616761418 - 4829.194027149335) / 4592.687616761418 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.10 μs 3.73 μs 1.20 0.16 False
1.73 μs 1.93 μs 1.12 0.07 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.7323516390342166 > 3.338203501307552. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.939743798307223 (T) = (0 -3710.0290310293003) / Math.Sqrt((12173.343555695912 / (51)) + (782.9885736938168 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.15539525340001847 = (3211.0474922860203 - 3710.0290310293003) / 3211.0474922860203 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9340360617337105 > 1.7100080295649096. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -38.17008352813366 (T) = (0 -1951.5449288019527) / Math.Sqrt((1782.4019270592116 / (50)) + (182.36451482474072 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.21401926377130706 = (1607.5073823289663 - 1951.5449288019527) / 1607.5073823289663 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<StructRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.73 μs 6.11 μs 1.29 0.07 True
2.46 μs 2.87 μs 1.17 0.15 False
3.74 μs 4.49 μs 1.20 0.10 True
3.06 μs 3.96 μs 1.29 0.09 False
3.44 μs 4.18 μs 1.22 0.10 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.105191257469472 > 5.117922017712586. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.649219500468597 (T) = (0 -5997.653248072635) / Math.Sqrt((12129.574995762472 / (49)) + (32091.510190343568 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2373158406861036 = (4847.30983864789 - 5997.653248072635) / 4847.30983864789 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.8692556411667587 > 2.5935605904987025. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 8/21/2023 10:38:45 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.85100681776673 (T) = (0 -2865.579621525271) / Math.Sqrt((6073.139650888221 / (50)) + (3801.56784736852 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.1415167380898485 = (2510.3264156427304 - 2865.579621525271) / 2510.3264156427304 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.4871946708501635 > 3.8419189952978097. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -37.71453321761238 (T) = (0 -4506.3724497052235) / Math.Sqrt((7658.286245019751 / (50)) + (1539.3998726272157 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.24088562164764124 = (3631.577617703142 - 4506.3724497052235) / 3631.577617703142 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.956616296694494 > 3.2545561442675495. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.456788531505822 (T) = (0 -3902.566503885601) / Math.Sqrt((7596.947598951103 / (51)) + (3783.409976522034 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2482502724129579 = (3126.429523096885 - 3902.566503885601) / 3126.429523096885 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.1844184444075605 > 3.561797373008362. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -56.05066729678553 (T) = (0 -4202.520471525489) / Math.Sqrt((6379.141545997949 / (51)) + (312.1763034432311 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.23470086711919388 = (3403.674998083395 - 4202.520471525489) / 3403.674998083395 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
102.00 μs 125.87 μs 1.23 0.07 False
90.68 μs 108.58 μs 1.20 0.06 False
210.89 μs 242.46 μs 1.15 0.07 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 125.86596946404568 > 106.97269076894881. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -39.42491893089427 (T) = (0 -125084.53938695304) / Math.Sqrt((8584828.776511228 / (51)) + (502947.02341180615 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.20615464035985312 = (103705.22584868089 - 125084.53938695304) / 103705.22584868089 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 108.58104699074076 > 95.62407202730078. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.046595045951197 (T) = (0 -107095.49506127542) / Math.Sqrt((4567296.624448414 / (50)) + (1692690.528780859 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.1579945662645974 = (92483.59032179129 - 107095.49506127542) / 92483.59032179129 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 242.45882451923077 > 229.1183659511657. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.761508767883058 (T) = (0 -242834.49665447537) / Math.Sqrt((29606711.37463386 / (52)) + (6447844.511772305 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.1135165516862032 = (218078.92867577943 - 242834.49665447537) / 218078.92867577943 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
421.60 μs 594.98 μs 1.41 0.23 True
9.77 μs 11.26 μs 1.15 0.30 False
63.94 μs 71.37 μs 1.12 0.33 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IndexerSet<Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 594.9752976190476 > 447.9053588425806. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -35.19285759460369 (T) = (0 -584105.4467372134) / Math.Sqrt((150924290.85646617 / (48)) + (81752075.92933121 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.41360287113887106 = (413203.3534047848 - 584105.4467372134) / 413203.3534047848 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IndexerSet<Int32>.IList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.261415525114156 > 10.253538318562121. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/19/2023 8:08:41 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.954470059674341 (T) = (0 -11191.243101541264) / Math.Sqrt((118695.28007347535 / (11)) + (478247.93028396316 / (42))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (42) - 2, .025) and -0.11837959188075431 = (10006.65890435393 - 11191.243101541264) / 10006.65890435393 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IndexerSet<Int32>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 71.37145301288324 > 66.05805536702154. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.618957925025676 (T) = (0 -69116.1706080707) / Math.Sqrt((9258286.992425486 / (39)) + (2128309.437222693 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.10604247641627756 = (62489.616883445684 - 69116.1706080707) / 62489.616883445684 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.37 μs 6.73 μs 1.25 0.10 False
4.37 μs 5.03 μs 1.15 0.09 False
9.27 μs 10.98 μs 1.19 0.11 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.73098335056797 > 5.565624551288168. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -24.655036062406715 (T) = (0 -6613.0039839743295) / Math.Sqrt((17782.182926717593 / (48)) + (10316.141387310783 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.2538861270994685 = (5274.006818523268 - 6613.0039839743295) / 5274.006818523268 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.034981543933963 > 4.622281808934856. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -6.485152239573478 (T) = (0 -4777.8882302460115) / Math.Sqrt((9416.695654317708 / (38)) + (48318.83520665564 / (14))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (14) - 2, .025) and -0.08998178093590795 = (4383.456965806804 - 4777.8882302460115) / 4383.456965806804 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromString<Location>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.983363006961259 > 10.013559975411276. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 10:37:55 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -20.399365062614947 (T) = (0 -10927.914803201556) / Math.Sqrt((108094.8052876687 / (49)) + (13976.291464311376 / (5))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (5) - 2, .025) and -0.15209219990636128 = (9485.278004737595 - 10927.914803201556) / 9485.278004737595 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
156.63 μs 239.30 μs 1.53 0.09 False
432.64 μs 611.44 μs 1.41 0.21 True
107.65 μs 121.18 μs 1.13 0.21 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsKeyTrue<Int32, Int32>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 239.29507449494952 > 175.4619518972432. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -102.88702967189477 (T) = (0 -240175.04653152436) / Math.Sqrt((16394861.157998888 / (51)) + (598742.6949682663 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.4476453994147958 = (165907.37388355882 - 240175.04653152436) / 165907.37388355882 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsKeyTrue<Int32, Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 611.4448936760356 > 453.93653982645213. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.276623426229268 (T) = (0 -620624.5701725839) / Math.Sqrt((128427139.44151728 / (51)) + (352826829.65940565 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.4201432207928535 = (437015.479203636 - 620624.5701725839) / 437015.479203636 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 121.18190254120879 > 113.709764022315. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -8.131385599928691 (T) = (0 -118654.32186470486) / Math.Sqrt((9323975.060975105 / (39)) + (13650595.768304855 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.0816789145305289 = (109694.58706348483 - 118654.32186470486) / 109694.58706348483 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.Logging.Formatting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.74 μs 1.97 μs 1.13 0.11 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.Formatting*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.Logging.Formatting.TwoArguments #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9701199997501861 > 1.7984912285404913. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.789720400977407 (T) = (0 -1933.7455430414316) / Math.Sqrt((1208.2703066409745 / (50)) + (1583.666193362613 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.12912821170113456 = (1712.6005027613894 - 1933.7455430414316) / 1712.6005027613894 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.94 ms 3.68 ms 1.25 0.05 False
1.47 ms 1.68 ms 1.14 0.04 False
2.25 ms 2.82 ms 1.25 0.08 False
1.51 ms 1.82 ms 1.21 0.06 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.675830088744589 > 3.0858829774869982. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -25.59205352889404 (T) = (0 -3660517.444609386) / Math.Sqrt((4162240985.2258677 / (51)) + (2478966216.903816 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.22722462330266546 = (2982760.7555317176 - 3660517.444609386) / 2982760.7555317176 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6826599929166663 > 1.5512105342427884. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -40.001775328522314 (T) = (0 -1691745.7017245372) / Math.Sqrt((753461474.4214141 / (51)) + (59913337.302220665 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.1480716467614428 = (1473554.1170246007 - 1691745.7017245372) / 1473554.1170246007 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.818698683908046 > 2.397773844537037. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.881311166977122 (T) = (0 -2741313.562856422) / Math.Sqrt((3603109158.8468213 / (51)) + (3506921328.4281783 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.18463716685116396 = (2314053.314858419 - 2741313.562856422) / 2314053.314858419 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8206987567287782 > 1.5843193093778232. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -25.660275520540978 (T) = (0 -1824755.4211917531) / Math.Sqrt((1108383921.7649317 / (50)) + (354128524.6348211 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.19977810246924502 = (1520910.7562775582 - 1824755.4211917531) / 1520910.7562775582 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Linq.Tests.Perf_OrderBy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.41 ms 2.00 ms 1.42 0.06 True
1.03 ms 1.47 ms 1.42 0.06 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_OrderBy*'
### Payloads [Baseline]() [Compare]() ### System.Linq.Tests.Perf_OrderBy.OrderByValueType(NumberOfPeople: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.003249478125 > 1.4655821045482773. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -35.929762828388164 (T) = (0 -1966278.9632490557) / Math.Sqrt((1199169307.584851 / (48)) + (880467791.3398414 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.4007081962724696 = (1403774.8679429942 - 1966278.9632490557) / 1403774.8679429942 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_OrderBy.OrderByCustomComparer(NumberOfPeople: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4698527410037878 > 1.0823728970214845. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -25.791140667125404 (T) = (0 -1484899.2707566738) / Math.Sqrt((511432152.8352613 / (51)) + (960377610.6572028 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.4611257903807798 = (1016270.6596053571 - 1484899.2707566738) / 1016270.6596053571 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
486.25 μs 683.95 μs 1.41 0.17 True
103.75 μs 113.23 μs 1.09 0.19 False
177.35 μs 241.55 μs 1.36 0.10 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsKeyFalse<Int32, Int32>.SortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 683.9506458333334 > 508.1515851978279. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -19.33437073781925 (T) = (0 -665405.655328028) / Math.Sqrt((186492104.0344951 / (51)) + (327316936.94410306 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.3673390778864137 = (486642.7545949974 - 665405.655328028) / 486642.7545949974 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 113.22980070558937 > 108.55767890494761. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/24/2023 11:15:45 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -8.493355920506499 (T) = (0 -112929.7298870898) / Math.Sqrt((14887434.42701175 / (39)) + (6720576.504166572 / (14))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (14) - 2, .025) and -0.07505831488267974 = (105045.21319795915 - 112929.7298870898) / 105045.21319795915 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsKeyFalse<Int32, Int32>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 241.54628102564104 > 172.88086920556188. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -29.36037558162874 (T) = (0 -242385.8009301677) / Math.Sqrt((18395817.696193747 / (50)) + (25284122.085273046 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4561985482044929 = (166451.066188077 - 242385.8009301677) / 166451.066188077 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
22.80 ms 25.00 ms 1.10 0.01 False
1.79 ms 2.26 ms 1.26 0.06 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.995733299999998 > 23.998217563504276. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.992324290101179 (T) = (0 -24747642.24574176) / Math.Sqrt((100975847082.62068 / (49)) + (76872055924.15459 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.07606428036893109 = (22998293.593814835 - 24747642.24574176) / 22998293.593814835 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.2574042018018017 > 1.9052025554528986. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -79.33620145917632 (T) = (0 -2262620.033013399) / Math.Sqrt((1196005273.4450774 / (50)) + (23283230.295007035 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2364326085815518 = (1829958.234124139 - 2262620.033013399) / 1829958.234124139 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
282.29 μs 322.08 μs 1.14 0.05 False
278.86 μs 329.52 μs 1.18 0.04 False
272.63 μs 335.45 μs 1.23 0.03 False
289.62 μs 340.78 μs 1.18 0.04 False
285.02 μs 340.32 μs 1.19 0.01 False
275.04 μs 314.75 μs 1.14 0.05 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 322.08375892857146 > 294.23512549107147. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.823207994857652 (T) = (0 -326928.2168357569) / Math.Sqrt((60319473.12261521 / (50)) + (31798007.43465819 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.17157894997384682 = (279049.2410631438 - 326928.2168357569) / 279049.2410631438 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 329.52027759308504 > 290.8511590887525. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.146731423702969 (T) = (0 -329592.16596485145) / Math.Sqrt((44371471.16557682 / (51)) + (45459962.72541628 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17665728397975763 = (280108.8902030045 - 329592.16596485145) / 280108.8902030045 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 335.4461881205674 > 289.0280026510511. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.012423837892294 (T) = (0 -327130.78522237187) / Math.Sqrt((59182952.52620558 / (49)) + (42459078.35853457 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.17271989416308298 = (278950.48668533954 - 327130.78522237187) / 278950.48668533954 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 340.77998835403736 > 303.40029436921293. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.299704651241502 (T) = (0 -341346.5862686391) / Math.Sqrt((49335696.799134314 / (52)) + (10152865.742032247 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.16805132986996849 = (292235.9467769614 - 341346.5862686391) / 292235.9467769614 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 340.3178688888889 > 304.4043580034723. IsChangePoint: Marked as a change because one of 9/26/2023 10:37:55 AM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -38.70081981533558 (T) = (0 -340490.02209342655) / Math.Sqrt((39040083.30638142 / (50)) + (3828426.629560999 / (5))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (5) - 2, .025) and -0.16461583171463637 = (292362.52231959713 - 340490.02209342655) / 292362.52231959713 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 314.75118467565596 > 290.28716847771705. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.81129294507482 (T) = (0 -323957.6482849186) / Math.Sqrt((46951640.23398351 / (48)) + (37702782.25499902 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.1594375451302722 = (279409.31328777986 - 323957.6482849186) / 279409.31328777986 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Activator<EmptyStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
277.53 ns 350.58 ns 1.26 0.12 True
351.24 ns 434.39 ns 1.24 0.08 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Activator&lt;EmptyStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Reflection.Activator<EmptyStruct>.CreateInstanceGeneric #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 350.57538511034824 > 287.965885866338. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.160300007367674 (T) = (0 -353.54964811010575) / Math.Sqrt((48.42915931572142 / (51)) + (25.721615754815403 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.26373533870241694 = (279.7655785056425 - 353.54964811010575) / 279.7655785056425 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Activator<EmptyStruct>.CreateInstanceType #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 434.38870313864334 > 371.7103634101547. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.54012763679372 (T) = (0 -432.0587791315115) / Math.Sqrt((79.25680954220935 / (50)) + (35.80389040021502 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.20385865775807463 = (358.8949386600975 - 432.0587791315115) / 358.8949386600975 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
96.32 μs 112.12 μs 1.16 0.02 False
156.43 μs 188.49 μs 1.20 0.08 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 112.12409569506728 > 99.69098308833881. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.43296311868252 (T) = (0 -111589.02407283173) / Math.Sqrt((2605324.1023553633 / (48)) + (851911.898877866 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.17682343920318166 = (94822.23106330022 - 111589.02407283173) / 94822.23106330022 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 188.4875869448903 > 165.19627077171077. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -25.663875529344438 (T) = (0 -186698.6470838832) / Math.Sqrt((21571165.33650393 / (50)) + (2724335.7826773855 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.16957442717086363 = (159629.55648363224 - 186698.6470838832) / 159629.55648363224 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.99 μs 4.96 μs 1.24 0.11 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;ClassImplementingIXmlSerialiable&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.955697201038025 > 4.348294075156738. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.954078901312158 (T) = (0 -4987.977678153509) / Math.Sqrt((13051.096827446157 / (51)) + (6106.928235363593 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.19109845224042016 = (4187.7123329068845 - 4987.977678153509) / 4187.7123329068845 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Devirtualization.EqualityComparer

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
346.07 ns 415.54 ns 1.20 0.14 False
362.65 ns 451.36 ns 1.24 0.16 False
329.78 ns 425.82 ns 1.29 0.13 True
326.20 ns 417.09 ns 1.28 0.06 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Devirtualization.EqualityComparer*'
### Payloads [Baseline]() [Compare]() ### Devirtualization.EqualityComparer.ValueTupleCompareNoOpt #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 415.54376076077494 > 357.9881410143185. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -30.16585283148647 (T) = (0 -421.784608628558) / Math.Sqrt((96.84220107073219 / (51)) + (21.839080658861445 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.24070980429132224 = (339.95428033993494 - 421.784608628558) / 339.95428033993494 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Devirtualization.EqualityComparer.ValueTupleCompareWrapped #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 451.36018724011717 > 378.07403913122323. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -30.226122663144153 (T) = (0 -446.7867121279275) / Math.Sqrt((115.53542213971389 / (50)) + (15.72822894490603 / (3))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (3) - 2, .025) and -0.2283991089274323 = (363.71461757086104 - 446.7867121279275) / 363.71461757086104 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Devirtualization.EqualityComparer.ValueTupleCompare #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 425.8212968293487 > 352.5853127863135. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -38.648110875965905 (T) = (0 -424.1975786369058) / Math.Sqrt((98.14586785718605 / (49)) + (10.948601347534007 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.24744344250427014 = (340.053556083729 - 424.1975786369058) / 340.053556083729 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Devirtualization.EqualityComparer.ValueTupleCompareCached #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 417.09095840493785 > 347.3086270870023. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -62.43798767002045 (T) = (0 -417.5862681037397) / Math.Sqrt((58.58603231554622 / (51)) + (2.334040131898836 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.24500840263787077 = (335.4083934043945 - 417.5862681037397) / 335.4083934043945 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
224.68 μs 265.07 μs 1.18 0.10 False
159.48 μs 191.88 μs 1.20 0.07 False
123.37 μs 154.10 μs 1.25 0.09 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 265.0658883227176 > 234.7563087424584. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -37.317170069553256 (T) = (0 -265758.6867945879) / Math.Sqrt((35334900.93879297 / (51)) + (1494242.0006914067 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.16959687482825608 = (227222.4665730335 - 265758.6867945879) / 227222.4665730335 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 191.87980725520046 > 166.57387253222763. IsChangePoint: Marked as a change because one of 8/24/2023 10:59:30 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.009814311577752 (T) = (0 -195025.1571253953) / Math.Sqrt((19428071.482499324 / (50)) + (5015052.98648303 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.22557374579583253 = (159129.67929869835 - 195025.1571253953) / 159129.67929869835 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 154.1037939285715 > 130.21266984137097. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -7.906186696837685 (T) = (0 -148486.68069182392) / Math.Sqrt((7872768.069165785 / (51)) + (25960735.340025023 / (3))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (3) - 2, .025) and -0.18767915195710297 = (125022.5538161059 - 148486.68069182392) / 125022.5538161059 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Threading.Channels.Tests.UnboundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
307.52 ns 459.10 ns 1.49 0.27 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.UnboundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### System.Threading.Channels.Tests.UnboundedChannelPerfTests.TryWriteThenTryRead #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 459.104474210873 > 325.19922474254275. IsChangePoint: Marked as a change because one of 9/21/2023 9:38:47 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.936363280455781 (T) = (0 -467.6578847003093) / Math.Sqrt((511.77685270247747 / (50)) + (302.49943914195075 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.4203463934535756 = (329.2562200712166 - 467.6578847003093) / 329.2562200712166 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.00 ms 2.71 ms 1.35 0.11 True
641.83 μs 762.35 μs 1.19 0.12 False
484.69 μs 602.58 μs 1.24 0.10 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.7112761904761897 > 2.0429815915655283. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -34.84728707977958 (T) = (0 -2663338.386945828) / Math.Sqrt((1709004124.5761523 / (51)) + (1458037578.3199513 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.35323959214886674 = (1968120.355329391 - 2663338.386945828) / 1968120.355329391 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 762.3481524347213 > 646.7839377066045. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -24.504524746268636 (T) = (0 -777621.5257143187) / Math.Sqrt((282386952.55952555 / (51)) + (147764755.86264217 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2584653008079083 = (617912.5679628211 - 777621.5257143187) / 617912.5679628211 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 602.5820755494506 > 493.064040426972. IsChangePoint: Marked as a change because one of 8/11/2023 7:16:47 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -40.37575028660534 (T) = (0 -601392.3031294516) / Math.Sqrt((163712431.31194684 / (48)) + (25079515.373683054 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.264043035939705 = (475768.8512419747 - 601392.3031294516) / 475768.8512419747 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
120.94 μs 155.23 μs 1.28 0.10 False
105.46 μs 138.70 μs 1.32 0.07 True
106.32 μs 136.01 μs 1.28 0.11 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;TreeRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromReader(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 155.22808017857145 > 128.68300274169923. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -33.78535347768506 (T) = (0 -153362.52680932276) / Math.Sqrt((7236672.326874238 / (49)) + (2652691.8159856778 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2474627075045832 = (122939.56836281561 - 153362.52680932276) / 122939.56836281561 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 138.69506173584298 > 111.50091870898576. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -20.02942754061871 (T) = (0 -136570.2862443796) / Math.Sqrt((7632563.12754147 / (51)) + (7532280.309398581 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2643807622212913 = (108013.57496491008 - 136570.2862443796) / 108013.57496491008 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 136.01224912207357 > 111.04249318380799. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.636930760360695 (T) = (0 -136553.18799083313) / Math.Sqrt((5792013.427429815 / (50)) + (15928575.799555605 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.27711767852760366 = (106922.94867319203 - 136553.18799083313) / 106922.94867319203 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.25 ms 2.84 ms 1.26 0.07 False
1.59 ms 1.84 ms 1.16 0.03 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.842760798172757 > 2.457038108738994. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -15.762105407880956 (T) = (0 -2799099.640924672) / Math.Sqrt((4696260405.748839 / (49)) + (3199975109.6364326 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.2027076594821721 = (2327331.682688235 - 2799099.640924672) / 2327331.682688235 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8421544962962964 > 1.6215417276306239. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -18.32417667707505 (T) = (0 -1877161.4657530622) / Math.Sqrt((663737136.3735952 / (52)) + (1137753138.9455152 / (4))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (52) + (4) - 2, .025) and -0.2023369696157834 = (1561260.7057678052 - 1877161.4657530622) / 1561260.7057678052 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
571.85 ns 708.47 ns 1.24 0.11 False
2.45 μs 3.43 μs 1.40 0.04 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.DependencyInjection.GetService.Scoped #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 708.4678965205258 > 599.5148870146672. IsChangePoint: Marked as a change because one of 8/14/2023 10:44:26 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.55805643174902 (T) = (0 -716.7337896521078) / Math.Sqrt((221.57267085423436 / (50)) + (135.68948350969754 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.24210686124477912 = (577.0306984165856 - 716.7337896521078) / 577.0306984165856 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Microsoft.Extensions.DependencyInjection.GetService.Transient #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.4317351365700057 > 2.540668420259039. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -60.94661624027439 (T) = (0 -3465.0072817273076) / Math.Sqrt((3630.6704472601728 / (49)) + (840.4007619025352 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.421504600886219 = (2437.5631845068197 - 3465.0072817273076) / 2437.5631845068197 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
401.42 ns 544.06 ns 1.36 0.19 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.TryWriteThenTryRead #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 544.0624314143822 > 418.10380759038094. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -27.90635303268526 (T) = (0 -546.9168605974861) / Math.Sqrt((138.4779106587325 / (49)) + (107.05763839037947 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.384199179902735 = (395.114278738351 - 546.9168605974861) / 395.114278738351 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
250.56 μs 309.95 μs 1.24 0.07 False
83.89 μs 102.78 μs 1.23 0.03 False
148.35 μs 183.28 μs 1.24 0.06 False
77.40 μs 91.29 μs 1.18 0.05 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 309.9530246975385 > 263.81624589563745. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.520792328381916 (T) = (0 -311871.8330786524) / Math.Sqrt((58267001.582678735 / (51)) + (77516442.82695965 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.22229976693171152 = (255151.67515864896 - 311871.8330786524) / 255151.67515864896 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 102.78240398041385 > 90.11836385619178. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -46.93249226864946 (T) = (0 -102317.1184482423) / Math.Sqrt((2659813.898420149 / (49)) + (285428.4803405726 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.19415485970439156 = (85681.6162633802 - 102317.1184482423) / 85681.6162633802 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 183.2798733430799 > 154.7195874849443. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -12.503552120560467 (T) = (0 -177532.1548002376) / Math.Sqrt((11241407.913137902 / (48)) + (22454992.624787193 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.20527960336673567 = (147295.41120942633 - 177532.1548002376) / 147295.41120942633 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 91.29281691176469 > 83.31808089274702. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -16.329866199132095 (T) = (0 -93616.15053039262) / Math.Sqrt((3046838.0817533447 / (51)) + (2720622.1125726863 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.17653100298199326 = (79569.64184804012 - 93616.15053039262) / 79569.64184804012 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Tests.Perf_SortedSet

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
35.77 μs 45.97 μs 1.29 0.12 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_SortedSet*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Tests.Perf_SortedSet.EnumerateViewBetween #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 45.96517613636364 > 37.721931704653535. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -29.968901388678546 (T) = (0 -46247.46044166744) / Math.Sqrt((945617.675594004 / (51)) + (345790.26121319603 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.26577298787443165 = (36536.931096411834 - 46247.46044166744) / 36536.931096411834 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.51 μs 4.16 μs 1.18 0.05 False
5.74 μs 6.78 μs 1.18 0.10 False
7.43 μs 8.68 μs 1.17 0.08 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<Location>.SystemTextJson_Reflection_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.157648402650223 > 3.6880431543166994. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -40.524512430223695 (T) = (0 -4125.985682769535) / Math.Sqrt((5922.507941453579 / (51)) + (693.3523386831133 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.20063108034342325 = (3436.5141385390057 - 4125.985682769535) / 3436.5141385390057 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<Location>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.782444695224697 > 6.025702071690361. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -10.734380832615942 (T) = (0 -6970.074321826536) / Math.Sqrt((42518.31460675569 / (50)) + (43774.706704705015 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.200843263161144 = (5804.316462981402 - 6970.074321826536) / 5804.316462981402 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<Location>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.683840315849876 > 8.003169111921734. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -40.64972944171845 (T) = (0 -8676.804041927862) / Math.Sqrt((33393.23870006933 / (50)) + (138.33399939341948 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.14176975144885626 = (7599.434151165219 - 8676.804041927862) / 7599.434151165219 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.52 ms 3.08 ms 1.23 0.13 False
3.05 ms 4.20 ms 1.38 0.09 True
11.51 ms 12.87 ms 1.12 0.01 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0832303562499996 > 2.6304744281875787. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -13.178846014562854 (T) = (0 -3121713.168816773) / Math.Sqrt((3692367507.712571 / (50)) + (8673539406.863163 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2498523157642922 = (2497665.627724846 - 3121713.168816773) / 2497665.627724846 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.203136903520209 > 3.192600598177897. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -70.94493700723106 (T) = (0 -4183344.9753759606) / Math.Sqrt((6767203236.7554655 / (50)) + (482242060.5044317 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.37229151121412396 = (3048437.5522186095 - 4183344.9753759606) / 3048437.5522186095 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.865933347368419 > 12.044023451403847. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -21.52886425731566 (T) = (0 -12869167.881302213) / Math.Sqrt((18535534270.083176 / (49)) + (15073789358.862349 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.12073280133584828 = (11482815.409670275 - 12869167.881302213) / 11482815.409670275 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.61 μs 3.31 μs 1.27 0.13 False
3.38 μs 4.09 μs 1.21 0.08 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.JsonNet_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.3077785406680364 > 2.7975783835741743. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -22.15576205956138 (T) = (0 -3242.135774842575) / Math.Sqrt((6228.179323965448 / (51)) + (2138.573201964515 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.2123121064277627 = (2674.340838182302 - 3242.135774842575) / 2674.340838182302 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.DataContractJsonSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.0889537673974 > 3.6579671460748893. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -23.792510518704585 (T) = (0 -4051.6600992164203) / Math.Sqrt((11368.815949321788 / (50)) + (1087.7531163454532 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.15103713234145552 = (3520.008160791892 - 4051.6600992164203) / 3520.008160791892 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.34 ms 10.81 ms 1.16 0.03 False
4.02 ms 5.50 ms 1.37 0.09 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.809025974025971 > 9.815579503. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.21053679561432 (T) = (0 -10867405.397573855) / Math.Sqrt((29828913625.98409 / (48)) + (23955459165.172623 / (4))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (4) - 2, .025) and -0.14778864719143076 = (9468124.13954933 - 10867405.397573855) / 9468124.13954933 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<MyEventsListerViewModel>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.50200740888889 > 4.2339673261408075. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -43.815272115038304 (T) = (0 -5443462.882295767) / Math.Sqrt((15667212738.494448 / (49)) + (2801290240.372464 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.34603475380016424 = (4044073.057495451 - 5443462.882295767) / 4044073.057495451 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.64 μs 10.49 μs 1.21 0.12 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;ClassImplementingIXmlSerialiable&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<ClassImplementingIXmlSerialiable>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.49015046108028 > 9.382416725127841. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -11.924374730859702 (T) = (0 -10397.856560845174) / Math.Sqrt((98040.56014849625 / (49)) + (41497.84999309578 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.1462318453786398 = (9071.338056752737 - 10397.856560845174) / 9071.338056752737 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
323.00 μs 419.10 μs 1.30 0.10 True
299.70 μs 346.11 μs 1.15 0.05 False
786.63 μs 893.06 μs 1.14 0.05 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.DataContractSerializer_BinaryXml_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 419.1044698544699 > 341.60025691220847. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -32.24608768333953 (T) = (0 -411486.7365177518) / Math.Sqrt((40650242.09980125 / (50)) + (26885399.433013912 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.2740507124704955 = (322975.1629900533 - 411486.7365177518) / 322975.1629900533 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 346.1058762820512 > 312.3305966574677. IsChangePoint: Marked as a change because one of 8/22/2023 3:00:25 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -28.460711687048494 (T) = (0 -348799.88448259636) / Math.Sqrt((45623885.31413718 / (51)) + (7477968.30140682 / (4))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (4) - 2, .025) and -0.15694909930780385 = (301482.4806824098 - 348799.88448259636) / 301482.4806824098 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 893.0583712237582 > 827.1792987710337. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -14.351661746462263 (T) = (0 -888319.0138999769) / Math.Sqrt((110304630.55872983 / (49)) + (186939060.6304711 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.1274917432842993 = (787871.8573249768 - 888319.0138999769) / 787871.8573249768 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.CtorFromCollectionNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
185.93 μs 257.74 μs 1.39 0.16 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CtorFromCollectionNonGeneric<Int32>.Hashtable(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 257.7379099043716 > 201.37288797882502. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -26.835683239488016 (T) = (0 -261658.92742559095) / Math.Sqrt((42384677.25215534 / (49)) + (25569554.8491088 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.3817712837720071 = (189364.8612462877 - 261658.92742559095) / 189364.8612462877 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Interop.StructureToPtr

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
852.55 ns 1.09 μs 1.28 0.18 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Interop.StructureToPtr*'
### Payloads [Baseline]() [Compare]() ### Interop.StructureToPtr.MarshalPtrToStructure #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0882190361964785 > 885.3745221326658. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -29.922449729265242 (T) = (0 -1077.7955573495597) / Math.Sqrt((687.6206531503506 / (50)) + (165.10561583352214 / (4))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (4) - 2, .025) and -0.25936206284044755 = (855.8266039224884 - 1077.7955573495597) / 855.8266039224884 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fc5c29692fc1a92426b7d1ce8c501e7696062bb6
Compare 1be2db9f75035a01e1955a33b35743018b6d85a6
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.68 μs 1.99 μs 1.18 0.20 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9864880971215815 > 1.7657768884418295. IsChangePoint: Marked as a change because one of 8/22/2023 3:00:25 AM, 9/26/2023 3:02:05 PM, 9/27/2023 11:11:59 PM falls between 9/19/2023 8:08:41 AM and 9/27/2023 11:11:59 PM. IsRegressionStdDev: Marked as regression because -17.4731321317645 (T) = (0 -1981.1231869095236) / Math.Sqrt((2993.8071867506237 / (49)) + (854.0408550745024 / (4))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (49) + (4) - 2, .025) and -0.17117388391698674 = (1691.5704953082325 - 1981.1231869095236) / 1691.5704953082325 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
lewing commented 1 year ago

range is https://github.com/dotnet/runtime/compare/aec06846449c331532022b9c01874c80e5a35fc6...07e86f1ed0070ce2afc0ae4d6a2c1e763f86e20b

@SamMonoRT this one looks like it is worth figuring out

lewing commented 1 year ago

@radical were there v8 version changes?

radical commented 1 year ago

@radical were there v8 version changes?

No, that hasn't changed for few weeks.

radical commented 1 year ago

@radical were there v8 version changes?

No, that hasn't changed for few weeks.

err, that's true only for library tests! For perf pipeline, since no version change shows up in the diff, there was no version change for v8 used in perf runs.

matouskozak commented 1 year ago

I didn't see these kinds of regressions for Mono AOT-llvm Perf line. I will update this comment if something new shows up in this week Perf issues.

SamMonoRT commented 1 year ago

@kotlarmilos - any related regression on the interpreter lane?

kotlarmilos commented 1 year ago

No related regressions on the interpreter lane. There is https://github.com/dotnet/perf-autofiling-issues/issues/22539, but it has different range.

radekdoulik commented 1 year ago

@radical @lewing could this be https://github.com/dotnet/runtime/pull/91480? range is https://github.com/dotnet/runtime/compare/aec06846449c331532022b9c01874c80e5a35fc6...07e86f1ed0070ce2afc0ae4d6a2c1e763f86e20b maybe we are not using the "current" runtime for some reason?

or it can be the v8` version change as we suspected earlier

radical commented 1 year ago

@radical @lewing could this be dotnet/runtime#91480? range is dotnet/runtime@aec0684...07e86f1 maybe we are not using the "current" runtime for some reason?

That is possible. The net8/9 build is messed up on main right now.

radical commented 1 year ago

I compared the two builds.

@lewing is working on a patch that would make this coherent.

radekdoulik commented 1 year ago

Did https://github.com/dotnet/runtime/pull/93031 fixed it?

Screenshot 2023-10-11 at 14 59 31
radical commented 1 year ago

Did dotnet/runtime#93031 fixed it?

Yeah, it should have fixed using the correct workload.

lewing commented 1 year ago

It is surprising that net9 already has that much of win over net8

SamMonoRT commented 1 year ago

It is surprising that net9 already has that much of win over net8

Any particular benchmarks? I know @matouskozak had a PR affecting many of them, but don't know of any other improvements

matouskozak commented 1 year ago

It is surprising that net9 already has that much of win over net8

Any particular benchmarks? I know @matouskozak had a PR affecting many of them, but don't know of any other improvements

It was this PR https://github.com/dotnet/runtime/commit/169e22c8f9f00719d87f0674954fee688b556b4a, affecting scenarios where mini is used (llvm should be unaffected).

lewing commented 1 year ago

these are wasm benchmarks so it is all llvm, the regression and subsequent improvment is the difference between using the net8 wasm runtime and the net9 wasm runtime on the perf lane which is still targeting net8 which in theory but the two changes alter in practice