search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 22 Regressions on 8/9/2023 9:46:11 AM #20576

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 4415472b81adecf3a12b24f8c808da3f33de966b
Compare f465d33afd016e7f57b4b37e1efb10ebb074086d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.53 ns 11.31 ns 1.19 0.04 False
7.86 ns 10.69 ns 1.36 0.04 False
16.69 ns 19.83 ns 1.19 0.11 False
7.89 ns 11.50 ns 1.46 0.05 False
16.56 ns 19.22 ns 1.16 0.15 False
13.14 ns 17.09 ns 1.30 0.08 False
9.47 ns 12.37 ns 1.31 0.03 False
21.07 ns 23.75 ns 1.13 0.15 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'
### Payloads [Baseline]() [Compare]() ### System.Text.Perf_Ascii.ToLower_Bytes_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/26/2023 10:42:28 PM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -57.16424264986325 (T) = (0 -11.276859863487928) / Math.Sqrt((1.473833602246313E-05 / (6)) + (0.004677673872681616 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.1837842055476683 = (9.526111102547427 - 11.276859863487928) / 9.526111102547427 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Bytes_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/11/2023 1:19:27 PM, 6/22/2023 7:55:14 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -166.19043677859318 (T) = (0 -10.686900826178592) / Math.Sqrt((0.001346551870552071 / (6)) + (0.00032821593585296774 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.36027175391763366 = (7.856445445845594 - 10.686900826178592) / 7.856445445845594 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Bytes_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/14/2023 2:26:40 PM, 7/28/2023 2:45:53 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -45.95293113043207 (T) = (0 -19.927230100571606) / Math.Sqrt((0.003277535678712797 / (6)) + (0.023175312497618363 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.1990288633907873 = (16.619474900894797 - 19.927230100571606) / 16.619474900894797 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/11/2023 1:19:27 PM, 6/22/2023 7:55:14 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -41.529866165416564 (T) = (0 -11.318537641411073) / Math.Sqrt((0.00041633409377778515 / (6)) + (0.034489727288657 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.4414816545675156 = (7.852016434303458 - 11.318537641411073) / 7.852016434303458 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToLower_Bytes_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/14/2023 2:26:40 PM, 7/28/2023 2:45:53 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -7.608823071682609 (T) = (0 -19.589930891666192) / Math.Sqrt((0.6681385535471808 / (6)) + (0.4451122871380028 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.21045497525944135 = (16.183940164703284 - 19.589930891666192) / 16.183940164703284 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Bytes(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/14/2023 2:26:40 PM, 8/10/2023 8:47:51 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -14.283590430903685 (T) = (0 -17.030973995436188) / Math.Sqrt((0.47749346900074524 / (8)) + (0.005946946493487139 / (3))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (3) - 2, .025) and -0.2630603968415904 = (13.483895178745096 - 17.030973995436188) / 13.483895178745096 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Bytes_Chars(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/26/2023 10:42:28 PM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -43.42395978200556 (T) = (0 -12.298925764299181) / Math.Sqrt((0.0006989315178336772 / (6)) + (0.020136947768545393 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.2941337805542388 = (9.503596883957329 - 12.298925764299181) / 9.503596883957329 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.ToUpper_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/17/2023 10:23:26 PM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -8.014358035642681 (T) = (0 -23.605204585620815) / Math.Sqrt((0.9355385430231913 / (6)) + (0.04419812163398953 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.15984130312295275 = (20.35209862078732 - 23.605204585620815) / 20.35209862078732 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 4415472b81adecf3a12b24f8c808da3f33de966b
Compare f465d33afd016e7f57b4b37e1efb10ebb074086d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.48 ns 11.65 ns 1.11 0.06 False
49.45 ns 58.23 ns 1.18 0.06 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
### Payloads [Baseline]() [Compare]() ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: -2147483648) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/30/2023 8:34:57 AM, 7/13/2023 3:06:28 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -25.971043571209435 (T) = (0 -11.76030859507308) / Math.Sqrt((0.0006480252633216109 / (6)) + (0.01191112115558628 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.12385083854671236 = (10.464296676844334 - 11.76030859507308) / 10.464296676844334 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 6/8/2023 8:02:22 AM, 7/1/2023 6:36:20 PM, 8/9/2023 9:46:11 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -8.630635753741645 (T) = (0 -61.08679895772873) / Math.Sqrt((0.506406254537837 / (7)) + (7.41138754645515 / (4))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (4) - 2, .025) and -0.2438343152186869 = (49.11168490072462 - 61.08679895772873) / 49.11168490072462 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 4415472b81adecf3a12b24f8c808da3f33de966b
Compare f465d33afd016e7f57b4b37e1efb10ebb074086d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
16.77 ns 21.29 ns 1.27 0.06 False
10.06 ns 14.55 ns 1.45 0.10 False
33.95 ns 37.03 ns 1.09 0.05 False
13.98 ns 17.31 ns 1.24 0.04 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Version.TryFormat4 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/2/2023 6:53:36 PM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -14.47383413382159 (T) = (0 -20.544687335911995) / Math.Sqrt((0.1675176228301793 / (6)) + (0.1853028501053935 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.21889803713222122 = (16.855132020926693 - 20.544687335911995) / 16.855132020926693 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Version.TryFormat2 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 6/10/2023 11:38:35 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -13.928442705416241 (T) = (0 -14.642191274119657) / Math.Sqrt((0.429868796471277 / (6)) + (0.016736903279973805 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.35226147710735317 = (10.82792900781366 - 14.642191274119657) / 10.82792900781366 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Version.TryFormatL #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/15/2023 6:46:12 PM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -6.3320443449428225 (T) = (0 -38.25404615315402) / Math.Sqrt((0.01671377551245345 / (6)) + (2.187551684452864 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.12338665284307601 = (34.0524307070414 - 38.25404615315402) / 34.0524307070414 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Version.TryFormat3 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/2/2023 6:53:36 PM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -21.21788543280975 (T) = (0 -18.055079864363414) / Math.Sqrt((0.0006480419082628464 / (6)) + (0.18493583712245334 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.2925575267812818 = (13.968492303258683 - 18.055079864363414) / 13.968492303258683 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 4415472b81adecf3a12b24f8c808da3f33de966b
Compare f465d33afd016e7f57b4b37e1efb10ebb074086d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
16.18 ns 20.40 ns 1.26 0.02 False
5.39 ns 8.09 ns 1.50 0.06 False
5.42 ns 8.82 ns 1.63 0.04 False
16.19 ns 19.41 ns 1.20 0.02 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/25/2023 12:29:35 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -33.4201466634763 (T) = (0 -20.691257434159024) / Math.Sqrt((0.003981056512298851 / (6)) + (0.08674781946996624 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.27677734876506893 = (16.205846269259183 - 20.691257434159024) / 16.205846269259183 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 12345) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/25/2023 12:29:35 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -254.79167750492726 (T) = (0 -8.097453384482588) / Math.Sqrt((0.00013016099378307405 / (6)) + (0.00046025365007729016 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.5050815405615583 = (5.380076206011643 - 8.097453384482588) / 5.380076206011643 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 12345) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/25/2023 12:29:35 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -229.2731435374104 (T) = (0 -8.788282642089134) / Math.Sqrt((0.00032827755870147454 / (6)) + (0.0008202000147926724 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.6282846213792516 = (5.3972644135427315 - 8.788282642089134) / 5.3972644135427315 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/25/2023 12:29:35 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -117.66017807780027 (T) = (0 -19.488423816013132) / Math.Sqrt((0.0022526915775650533 / (6)) + (0.0019885147227248325 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.20174200908943782 = (16.216811652260994 - 19.488423816013132) / 16.216811652260994 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 4415472b81adecf3a12b24f8c808da3f33de966b
Compare f465d33afd016e7f57b4b37e1efb10ebb074086d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
15.00 ns 23.40 ns 1.56 0.07 False
  • [IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0%e3%81%a0.html>)
  • 📝 - Benchmark Source
  • ADX - Test Multi Config Graph
 15.00 ns 20.78 ns 1.39 0.08 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/16/2023 12:30:38 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -8.656786291009206 (T) = (0 -22.23556383794095) / Math.Sqrt((0.0577614864970684 / (6)) + (3.340093206051046 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.4716412783831668 = (15.109364058046994 - 22.23556383794095) / 15.109364058046994 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/16/2023 12:30:38 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -12.790092112327645 (T) = (0 -22.630300704223586) / Math.Sqrt((7.5679138956198105E-06 / (6)) + (1.7762385554010707 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.5079777608401167 = (15.00705202152047 - 22.630300704223586) / 15.00705202152047 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 4415472b81adecf3a12b24f8c808da3f33de966b
Compare f465d33afd016e7f57b4b37e1efb10ebb074086d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
15.09 ns 20.02 ns 1.33 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Int32.TryFormat(value: -2147483648) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/13/2023 1:49:08 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -72.68136856598487 (T) = (0 -20.185946005144693) / Math.Sqrt((0.017382356065320827 / (6)) + (0.008996734933545728 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.3275801442701313 = (15.205067725867801 - 20.185946005144693) / 15.205067725867801 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 4415472b81adecf3a12b24f8c808da3f33de966b
Compare f465d33afd016e7f57b4b37e1efb10ebb074086d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
38.67 μs 53.07 μs 1.37 0.36 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/29/2023 9:06:42 PM, 6/5/2023 10:43:22 AM, 6/13/2023 5:43:42 PM, 6/29/2023 3:17:38 AM, 8/8/2023 9:37:01 AM, 8/14/2023 10:44:26 PM falls between 8/5/2023 9:09:13 PM and 8/14/2023 10:44:26 PM. IsRegressionStdDev: Marked as regression because -5.97462760442861 (T) = (0 -51345.825) / Math.Sqrt((851355.3694068186 / (6)) + (21686550.493914794 / (5))) is less than -2.2621571627982036 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (5) - 2, .025) and -0.3267295985351245 = (38701.04733978364 - 51345.825) / 38701.04733978364 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)