search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 30 Regressions on 10/11/2023 7:15:38 PM #23290

Open performanceautofiler[bot] opened 12 months ago

performanceautofiler[bot] commented 12 months ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
27.88 μs 55.73 μs 2.00 0.04 False
27.85 μs 55.71 μs 2.00 0.04 False
27.85 μs 55.72 μs 2.00 0.03 False
27.85 μs 55.71 μs 2.00 0.04 False
27.86 μs 55.69 μs 2.00 0.04 False
27.88 μs 55.69 μs 2.00 0.04 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceNo #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.726525800711755 > 29.248739369086795. IsChangePoint: Marked as a change because one of 6/21/2023 9:03:14 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -3538.4029893574516 (T) = (0 -55716.23136925671) / Math.Sqrt((77.89097330006285 / (19)) + (636.7376582658295 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.9999909774375648 = (27858.241360989363 - 55716.23136925671) / 27858.241360989363 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.CheckObjIsInterfaceNo #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.70523322318251 > 29.236794150355877. IsChangePoint: Marked as a change because one of 6/21/2023 3:14:03 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -5751.0326393319665 (T) = (0 -55702.86290637141) / Math.Sqrt((82.54549200428252 / (19)) + (210.15422109958863 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.9998362840209527 = (27853.711502010032 - 55702.86290637141) / 27853.711502010032 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceYes #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.71524847483477 > 29.24311363636364. IsChangePoint: Marked as a change because one of 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -7908.73895058744 (T) = (0 -55700.72290664691) / Math.Sqrt((76.42967893954408 / (19)) + (92.10148310988265 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.9995765639333603 = (27856.2591257212 - 55700.72290664691) / 27856.2591257212 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.CheckObjIsInterfaceYes #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.709246314184035 > 29.249876851090086. IsChangePoint: Marked as a change because one of 6/20/2023 3:53:12 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -4330.8509348133575 (T) = (0 -55701.121598011494) / Math.Sqrt((61.5898678955438 / (19)) + (419.0339859287466 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.9995454424399719 = (27856.892079452547 - 55701.121598011494) / 27856.892079452547 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.CheckArrayIsInterfaceYes #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.69166826923077 > 29.25336296791444. IsChangePoint: Marked as a change because one of 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -3971.2408479026967 (T) = (0 -55710.2198868403) / Math.Sqrt((271.5083994977357 / (19)) + (383.70842521567073 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.9994590842150066 = (27862.64561583279 - 55710.2198868403) / 27862.64561583279 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.CastingPerf.CheckArrayIsInterfaceNo #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.69481162742951 > 29.267872855392163. IsChangePoint: Marked as a change because one of 6/19/2023 11:51:49 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -3768.218956167211 (T) = (0 -55710.46312450895) / Math.Sqrt((98.29667071878303 / (19)) + (543.6879320995562 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.9991919324622408 = (27866.490565463086 - 55710.46312450895) / 27866.490565463086 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Perf_LengthBucketsFrozenDictionary

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.25 μs 1.75 μs 1.40 0.52 False
1.21 μs 1.51 μs 1.25 0.32 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_LengthBucketsFrozenDictionary*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Perf_LengthBucketsFrozenDictionary.ToFrozenDictionary(Count: 100, ItemsPerBucket: 5) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7479644966166217 > 1.3954740980320892. IsChangePoint: Marked as a change because one of 9/11/2023 8:56:48 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -3.5423077225832156 (T) = (0 -1475.4266304236735) / Math.Sqrt((6937.3244555081965 / (19)) + (14033.912741072107 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.10773310394376451 = (1331.9333196514956 - 1475.4266304236735) / 1331.9333196514956 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_LengthBucketsFrozenDictionary.ToFrozenDictionary(Count: 100, ItemsPerBucket: 1) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.512324609594165 > 1.2732832576524191. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 10/14/2023 5:24:18 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -2.230527627331502 (T) = (0 -1412.231962806666) / Math.Sqrt((3388.904915886191 / (27)) + (15248.721282670545 / (3))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (3) - 2, .025) and -0.12865138807668403 = (1251.2561254305697 - 1412.231962806666) / 1251.2561254305697 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.GetMember

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
58.88 μs 79.44 μs 1.35 0.03 True
903.70 μs 1.38 ms 1.53 0.02 False
313.30 μs 403.15 μs 1.29 0.03 True
126.51 μs 158.75 μs 1.25 0.02 True
257.32 μs 315.77 μs 1.23 0.03 True
176.50 μs 239.21 μs 1.36 0.05 False
606.47 μs 793.94 μs 1.31 0.02 True
1.25 ms 1.62 ms 1.30 0.03 True
753.94 μs 956.83 μs 1.27 0.04 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.GetMember*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.GetMember.GetMethod1 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 79.44238491430849 > 63.26769462497106. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -57.604544581433785 (T) = (0 -79922.09473909302) / Math.Sqrt((1473808.4267497093 / (19)) + (392166.9136653599 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.3201714203891422 = (60539.1796131556 - 79922.09473909302) / 60539.1796131556 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod15 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3835830718623483 > 972.7890705599548. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -17.031334106305554 (T) = (0 -1216737.2843384685) / Math.Sqrt((133077128.83835538 / (19)) + (3131283573.3266706 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.3141526438506064 = (925872.1123699142 - 1216737.2843384685) / 925872.1123699142 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod5 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 403.1538125 > 325.22595765625005. IsChangePoint: Marked as a change because one of 6/29/2023 6:18:13 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -68.48688842856717 (T) = (0 -400017.72083818953) / Math.Sqrt((21024073.756303746 / (19)) + (6697114.424145451 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.2890521267325156 = (310319.2745604112 - 400017.72083818953) / 310319.2745604112 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod2 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 158.74823453608246 > 132.84140831300815. IsChangePoint: Marked as a change because one of 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -79.3201532165294 (T) = (0 -159766.03156986897) / Math.Sqrt((1243610.9360891897 / (19)) + (1275546.5321656023 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.2681695026410759 = (125981.60674668645 - 159766.03156986897) / 125981.60674668645 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod4 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 315.7719553571428 > 267.70837500000005. IsChangePoint: Marked as a change because one of 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -76.91541938588284 (T) = (0 -317877.45111257787) / Math.Sqrt((6367530.508018525 / (19)) + (3804768.6121771624 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.24949980586819934 = (254403.76190511265 - 317877.45111257787) / 254403.76190511265 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod3 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 239.2114102564103 > 191.3317985983456. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -15.846865493622063 (T) = (0 -242072.4835412264) / Math.Sqrt((9668447.32408191 / (19)) + (157696261.61916476 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.3372988916594915 = (181015.9905545363 - 242072.4835412264) / 181015.9905545363 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod10 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 793.9358035714286 > 646.2802205528845. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -59.33666373025208 (T) = (0 -798908.6176011489) / Math.Sqrt((95797744.67976247 / (19)) + (50567431.299106814 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.29969907159968 = (614687.3803778638 - 798908.6176011489) / 614687.3803778638 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod20 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6201816517857144 > 1.3050890841346152. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -55.928156543631815 (T) = (0 -1623330.0903709389) / Math.Sqrt((652727166.0088447 / (19)) + (150296714.11912242 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.3136095495276562 = (1235778.2348298633 - 1623330.0903709389) / 1235778.2348298633 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.GetMember.GetMethod12 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 956.8304901960784 > 777.6610226449277. IsChangePoint: Marked as a change because one of 8/2/2023 5:07:17 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -46.116629302294065 (T) = (0 -963552.765849347) / Math.Sqrt((238597204.61019626 / (19)) + (130698679.00955084 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.30994140743603127 = (735569.3624002037 - 963552.765849347) / 735569.3624002037 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 12 months ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
576.94 ns 638.33 ns 1.11 0.05 False
753.99 ns 868.10 ns 1.15 0.04 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.IndexerBench*'
### Payloads [Baseline]() [Compare]() ### Span.IndexerBench.Ref(length: 1024) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 638.3254694646729 > 606.110519631975. IsChangePoint: Marked as a change because one of 6/30/2023 8:34:57 AM, 8/14/2023 10:44:26 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -351.1760862720962 (T) = (0 -638.5167980702851) / Math.Sqrt((0.10246949700553933 / (19)) + (0.2750823869925427 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.1060656454709476 = (577.2865296782754 - 638.5167980702851) / 577.2865296782754 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Span.IndexerBench.CoveredIndex2(length: 1024) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 868.102990094962 > 791.6596816666101. IsChangePoint: Marked as a change because one of 6/30/2023 8:34:57 AM, 8/14/2023 10:44:26 PM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -328.5651950556469 (T) = (0 -868.3911002765115) / Math.Sqrt((2.249410589125519 / (19)) + (0.041534757922182954 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.15239980059680802 = (753.5502000493116 - 868.3911002765115) / 753.5502000493116 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
275.66 ns 316.01 ns 1.15 0.33 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: None) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 316.0131588806692 > 291.19416583603964. IsChangePoint: Marked as a change because one of 8/18/2023 9:39:07 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -8.224386643686877 (T) = (0 -311.3767010351214) / Math.Sqrt((146.85673980398246 / (19)) + (107.40222065533501 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.12419141026014788 = (276.9783670229842 - 311.3767010351214) / 276.9783670229842 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
310.91 ms 340.71 ms 1.10 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_StreamWriter*'
### Payloads [Baseline]() [Compare]() ### System.IO.Tests.Perf_StreamWriter.WritePartialCharArray(writeLength: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 340.71332 > 327.42505950000003. IsChangePoint: Marked as a change because one of 7/17/2023 6:26:12 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -20.795750905276176 (T) = (0 -340860640.8275058) / Math.Sqrt((23613593189278.59 / (19)) + (4181497338613.5483 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.08427329908928989 = (314367826.9250048 - 340860640.8275058) / 314367826.9250048 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.37 μs 2.62 μs 1.10 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.TryGetValueFalse<Int32, Int32>.IDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.615038305322129 > 2.4834159804800517. IsChangePoint: Marked as a change because one of 7/1/2023 6:36:20 PM, 7/27/2023 4:03:50 PM, 9/1/2023 9:23:50 AM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -143.5057428244842 (T) = (0 -2613.107553079247) / Math.Sqrt((4.623030854123893 / (19)) + (29.873270238731333 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.10432448829606158 = (2366.249757905116 - 2613.107553079247) / 2366.249757905116 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.SequenceReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
71.29 ns 86.38 ns 1.21 0.34 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.SequenceReader*'
### Payloads [Baseline]() [Compare]() ### System.Memory.SequenceReader.TryReadTo #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 86.38168810283534 > 75.75985132249512. IsChangePoint: Marked as a change because one of 7/13/2023 3:01:51 PM, 7/14/2023 8:09:31 AM, 8/28/2023 1:23:07 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -5.2947365757186455 (T) = (0 -83.48512278480763) / Math.Sqrt((28.497248902005577 / (19)) + (13.321528188133101 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.11659740618480677 = (74.76743392236584 - 83.48512278480763) / 74.76743392236584 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
19.58 ns 22.44 ns 1.15 0.08 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
### Payloads [Baseline]() [Compare]() ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.43694348827947 > 20.497181381316913. IsChangePoint: Marked as a change because one of 8/14/2023 10:44:26 PM, 9/1/2023 10:04:35 PM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -19.966642191061005 (T) = (0 -22.406550092029732) / Math.Sqrt((0.03634700136457252 / (19)) + (0.20702124741946387 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.1471975239441473 = (19.53155374237072 - 22.406550092029732) / 19.53155374237072 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
556.39 ns 594.06 ns 1.07 0.17 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongMultiLine) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 594.0570899772209 > 584.2271798935185. IsChangePoint: Marked as a change because one of 8/14/2023 10:44:26 PM, 9/2/2023 3:43:46 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -78.95291393373002 (T) = (0 -593.0028092584203) / Math.Sqrt((1.8196778214216554 / (19)) + (1.3076823139448928 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.06574019107360368 = (556.4234268588876 - 593.0028092584203) / 556.4234268588876 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.48 μs 21.39 μs 1.22 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsFalse<Int32>.Array(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.38718365253078 > 18.35518367125113. IsChangePoint: Marked as a change because one of 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 7/23/2023 4:13:29 AM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -35.74719769683485 (T) = (0 -21523.15763139922) / Math.Sqrt((198199.68521044322 / (19)) + (11435.764555697784 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.21637525292482293 = (17694.50470128024 - 21523.15763139922) / 17694.50470128024 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
34.24 ns 45.68 ns 1.33 0.13 False
28.45 ns 35.73 ns 1.26 0.24 False
5.70 ns 7.24 ns 1.27 0.13 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 45.67563576024887 > 35.11607017053264. IsChangePoint: Marked as a change because one of 6/19/2023 6:37:24 PM, 7/5/2023 7:07:19 PM, 7/12/2023 9:48:15 PM, 8/19/2023 11:06:43 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -20.907189818231938 (T) = (0 -44.40689991376266) / Math.Sqrt((0.9773822036267911 / (19)) + (2.454894252540422 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.327529306258535 = (33.45078688990875 - 44.40689991376266) / 33.45078688990875 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.7334938939509 > 29.780808851429754. IsChangePoint: Marked as a change because one of 6/19/2023 6:37:24 PM, 8/19/2023 11:06:43 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -24.883747222466067 (T) = (0 -35.87834299741914) / Math.Sqrt((1.4496565265266763 / (19)) + (0.2085442339327479 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.2723575192471744 = (28.198318833096184 - 35.87834299741914) / 28.198318833096184 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.239332393207186 > 5.979298934073361. IsChangePoint: Marked as a change because one of 6/19/2023 6:37:24 PM, 7/5/2023 7:07:19 PM, 7/13/2023 3:06:28 AM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -43.95976692376355 (T) = (0 -7.241511340553896) / Math.Sqrt((0.01371912240795933 / (19)) + (0.005797649048170044 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.27315503074077 = (5.687847250103163 - 7.241511340553896) / 5.687847250103163 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 5f4ee519c7ce0bfe1e914ee7a617e17b5c509cf5
Compare b62445b0b3cd6d9d70a4017ff2d39c58fb830d93
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.96 μs 5.75 μs 1.16 0.10 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateForEach<Int32>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.754570404292063 > 5.198863986490865. IsChangePoint: Marked as a change because one of 6/19/2023 6:37:24 PM, 6/26/2023 10:42:28 PM, 7/23/2023 4:13:29 AM, 8/14/2023 10:44:26 PM, 9/19/2023 8:08:41 AM, 10/11/2023 2:37:28 PM, 10/16/2023 8:09:45 PM falls between 10/8/2023 1:17:11 AM and 10/16/2023 8:09:45 PM. IsRegressionStdDev: Marked as regression because -95.5289748452938 (T) = (0 -5763.68277346738) / Math.Sqrt((878.0525397174716 / (19)) + (292.6411912219055 / (11))) is less than -2.048407141790485 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (11) - 2, .025) and -0.16473181576654378 = (4948.506338924152 - 5763.68277346738) / 4948.506338924152 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)