search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/arm64: 31 Regressions on 7/18/2023 1:13:00 AM #20073

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
44.71 μs 153.58 μs 3.44 0.03 True
201.35 μs 234.08 μs 1.16 0.00 False
42.02 μs 73.88 μs 1.76 0.05 True
135.06 μs 161.57 μs 1.20 0.03 False
318.93 μs 364.03 μs 1.14 0.00 True
158.92 μs 207.86 μs 1.31 0.02 False
192.45 μs 252.88 μs 1.31 0.06 False
144.61 μs 189.63 μs 1.31 0.01 True
262.45 μs 304.30 μs 1.16 0.01 False
203.15 μs 238.28 μs 1.17 0.02 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'
### Payloads [Baseline]() [Compare]() ### System.Text.Perf_Utf8Encoding.GetString(Input: EnglishAllAscii) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.5777525 > 46.74993420038968. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -40.899271974966524 (T) = (0 -150439.03886212426) / Math.Sqrt((34701.155625382315 / (7)) + (100430799.75387879 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -2.3752536048258164 = (44571.18085794558 - 150439.03886212426) / 44571.18085794558 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 234.07882090336133 > 209.929507411859. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -68.56772817877568 (T) = (0 -237509.10568818776) / Math.Sqrt((1062959.9523671502 / (7)) + (2219739.1794010433 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.18776286474051826 = (199963.40409251186 - 237509.10568818776) / 199963.40409251186 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 73.87781250000002 > 44.28628649003249. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -21.748358179413767 (T) = (0 -77390.45704578982) / Math.Sqrt((255985.89785228716 / (7)) + (35938942.1218422 / (14))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (14) - 2, .025) and -0.8296582519337298 = (42297.76624350333 - 77390.45704578982) / 42297.76624350333 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 161.56669217687073 > 141.59178546238246. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -76.22829414061411 (T) = (0 -163679.75848820465) / Math.Sqrt((683093.8709460925 / (7)) + (626015.9675234444 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.21040583356928227 = (135227.17253067155 - 163679.75848820465) / 135227.17253067155 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: Greek) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 364.03040697674425 > 335.29442749215076. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -85.18549541748206 (T) = (0 -367294.1184767537) / Math.Sqrt((164789.3537792158 / (7)) + (4422028.622268606 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.15055966649166835 = (319230.8310239323 - 367294.1184767537) / 319230.8310239323 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 207.85948888888888 > 167.2544845328283. IsChangePoint: Marked as a change because one of 5/6/2023 11:12:53 AM, 5/19/2023 4:22:12 AM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -53.63105293840534 (T) = (0 -207857.74997291854) / Math.Sqrt((4146257.6842698413 / (7)) + (2770007.0649260827 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.2943898403477793 = (160583.57651901137 - 207857.74997291854) / 160583.57651901137 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: EnglishMostlyAscii) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 252.87836303710938 > 201.88532060944848. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -50.33581917158821 (T) = (0 -250394.7673829024) / Math.Sqrt((2534102.0045074294 / (7)) + (14401056.96743332 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.3006327343162242 = (192517.65758036324 - 250394.7673829024) / 192517.65758036324 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishMostlyAscii) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 189.63096762048195 > 151.97955061431625. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -104.21124656854758 (T) = (0 -191029.8138438226) / Math.Sqrt((258883.64683672794 / (7)) + (2402434.0298885005 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.3196413436788694 = (144758.8882834434 - 191029.8138438226) / 144758.8882834434 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: Chinese) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 304.30064388736264 > 275.58116796875. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -87.38669157554666 (T) = (0 -305987.4434228486) / Math.Sqrt((316335.76313669194 / (7)) + (3044511.444175744 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.16586480655495192 = (262455.3393347723 - 305987.4434228486) / 262455.3393347723 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: Greek) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 238.27921401515152 > 213.5713271103896. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -83.29108713322728 (T) = (0 -238215.718248004) / Math.Sqrt((375559.1036978655 / (7)) + (1832923.6054944617 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.17181156764697672 = (203288.41669172663 - 238215.718248004) / 203288.41669172663 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
197.19 μs 467.60 μs 2.37 0.31 True
281.38 μs 558.61 μs 1.99 0.26 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'
### Payloads [Baseline]() [Compare]() ### System.IO.Tests.Perf_File.ReadAllBytes(size: 1048576) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 467.59773437500013 > 207.55060158590933. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -31.3538860056875 (T) = (0 -481494.20531664253) / Math.Sqrt((338290917.26612204 / (7)) + (460787050.1675695 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -1.3750173163498958 = (202732.9240936394 - 481494.20531664253) / 202732.9240936394 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.Perf_File.ReadAllBytesAsync(size: 1048576) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 558.6136177884615 > 290.79834694157273. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -71.67841051676666 (T) = (0 -549478.315734813) / Math.Sqrt((15157470.726154061 / (7)) + (185696309.77354327 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.9900641877576938 = (276110.85065247974 - 549478.315734813) / 276110.85065247974 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
184.72 ns 812.85 ns 4.40 0.11 True
625.87 ns 2.18 μs 3.49 0.03 True
1.46 μs 2.21 μs 1.51 0.08 True
235.15 ns 843.88 ns 3.59 0.12 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 1000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 812.8530847222013 > 194.26384572489965. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -50.51488639780015 (T) = (0 -806.9452916378901) / Math.Sqrt((1.7529317528981885 / (7)) + (2266.6045402433742 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -3.3507232732335446 = (185.47382606528137 - 806.9452916378901) / 185.47382606528137 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 10000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1835266932360624 > 659.3037901588061. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -112.61480968726458 (T) = (0 -2255.1722210540393) / Math.Sqrt((38.40771117130947 / (7)) + (3042.0578190036467 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -2.5802982776535734 = (629.8838940681824 - 2255.1722210540393) / 629.8838940681824 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Char>.AllocateArray(length: 10000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.211189493002982 > 1.5407292667288688. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -55.71094592845798 (T) = (0 -2306.2793418728293) / Math.Sqrt((90.24203429337295 / (7)) + (3193.012564576641 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.5697449192102648 = (1469.2064383512152 - 2306.2793418728293) / 1469.2064383512152 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Char>.AllocateArray(length: 1000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 843.8797530917785 > 248.99258891410207. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -54.0886096220586 (T) = (0 -811.7868596106102) / Math.Sqrt((1.0412648447088642 / (7)) + (1691.7387669517434 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -2.42538000849053 = (236.9917666356505 - 811.7868596106102) / 236.9917666356505 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
14.25 μs 93.27 μs 6.55 0.03 True
277.65 μs 347.94 μs 1.25 0.49 False
21.28 μs 88.07 μs 4.14 0.08 True
5.44 μs 10.11 μs 1.86 0.05 True
19.13 μs 91.61 μs 4.79 0.03 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
### Payloads [Baseline]() [Compare]() ### System.Text.Tests.Perf_StringBuilder.ctor_string(length: 100000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 93.27147329650093 > 15.116572545453721. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -27.801559240638348 (T) = (0 -80711.79675214925) / Math.Sqrt((44505.920605590596 / (7)) + (84995920.23309411 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -4.568128513752843 = (14495.318589144883 - 80711.79675214925) / 14495.318589144883 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length: 100000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 347.9425328947368 > 296.2523920507812. IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 3/30/2023 5:15:39 PM, 5/18/2023 7:10:13 PM, 6/1/2023 5:52:46 AM, 6/28/2023 11:43:34 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -9.095730576520163 (T) = (0 -403714.9562237347) / Math.Sqrt((811387836.3838218 / (7)) + (900464765.5924665 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.42622293937282796 = (283065.81326006836 - 403714.9562237347) / 283065.81326006836 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 88.06867663043477 > 22.36466686417984. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -25.785898506328433 (T) = (0 -85938.12286982416) / Math.Sqrt((596593.0057647921 / (7)) + (92250964.2068706 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -2.9879823098854787 = (21549.273841260347 - 85938.12286982416) / 21549.273841260347 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.107941191817993 > 5.856456200510811. IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -160.51760678649475 (T) = (0 -10090.497115370224) / Math.Sqrt((2518.6455695499767 / (7)) + (6571.819651443418 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.816060064653965 = (5556.2573682236025 - 10090.497115370224) / 5556.2573682236025 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length: 100000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 91.60684881602916 > 20.131046706945337. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -28.332658953743195 (T) = (0 -82491.83107807054) / Math.Sqrt((42041.27955617552 / (7)) + (74623597.03170486 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -3.28326007724604 = (19259.12262864724 - 82491.83107807054) / 19259.12262864724 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
176.26 ns 444.52 ns 2.52 0.16 True
376.64 ns 1.86 μs 4.94 0.11 True
770.28 ns 1.81 μs 2.34 0.11 True
153.73 ns 426.35 ns 2.77 0.13 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_GC<Byte>.AllocateArray(length: 1000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 444.52469378473035 > 187.22435664952152. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -51.969766910058915 (T) = (0 -457.06757975331334) / Math.Sqrt((82.5092057131736 / (7)) + (245.98867902018307 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -1.5230578523671738 = (181.1562027103283 - 457.06757975331334) / 181.1562027103283 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 10000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8593231043150402 > 401.56605916469255. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -72.07839158795262 (T) = (0 -1754.18870186505) / Math.Sqrt((109.93100749718072 / (7)) + (5169.803334176505 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -3.545500921045185 = (385.9175770360849 - 1754.18870186505) / 385.9175770360849 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Byte>.AllocateArray(length: 10000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8056446688538865 > 813.7722789507309. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -54.93263873260624 (T) = (0 -1792.8236276370205) / Math.Sqrt((21.323454679243763 / (7)) + (5086.765274001457 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -1.3082644905685505 = (776.6976596323365 - 1792.8236276370205) / 776.6976596323365 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 1000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 426.3505707344087 > 162.68538993231937. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -43.815054128894985 (T) = (0 -462.338873787822) / Math.Sqrt((1.538664137652443 / (7)) + (685.7585040958033 / (14))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (14) - 2, .025) and -1.982827418797353 = (155.00020915532303 - 462.338873787822) / 155.00020915532303 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
138.72 μs 173.70 μs 1.25 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 173.7011563670412 > 145.6859375. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -31.61931374614034 (T) = (0 -171109.10813599516) / Math.Sqrt((108364.97428655898 / (7)) + (15515208.267456543 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.23356012275322607 = (138711.6079547795 - 171109.10813599516) / 138711.6079547795 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
326.08 μs 412.63 μs 1.27 0.20 False
372.40 μs 397.17 μs 1.07 0.21 False
292.14 μs 348.10 μs 1.19 0.18 False
349.54 μs 427.39 μs 1.22 0.23 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 412.62818627450974 > 338.0411594669117. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -7.685806969819311 (T) = (0 -362985.1518604368) / Math.Sqrt((81259375.86085501 / (7)) + (281121490.2842111 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.13205225639999946 = (320643.4595296452 - 362985.1518604368) / 320643.4595296452 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 397.1745416666667 > 374.9506987847222. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/14/2023 7:13:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -9.531406631209357 (T) = (0 -413282.668128477) / Math.Sqrt((134382005.97139543 / (6)) + (148837187.51943213 / (16))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (16) - 2, .025) and -0.1492243253331693 = (359618.7959288654 - 413282.668128477) / 359618.7959288654 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 348.0955370370371 > 332.7442472426471. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/14/2023 7:13:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -6.009668915564751 (T) = (0 -362096.4748151295) / Math.Sqrt((314135067.3246025 / (6)) + (194629092.80990002 / (16))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (6) + (16) - 2, .025) and -0.15381968648393918 = (313824.1434574186 - 362096.4748151295) / 313824.1434574186 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 427.3889660493827 > 369.0042809606482. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -8.65898692460267 (T) = (0 -413080.0048128988) / Math.Sqrt((314777463.60135657 / (7)) + (94726617.77410865 / (15))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (15) - 2, .025) and -0.17662828310015874 = (351070.94631834206 - 413080.0048128988) / 351070.94631834206 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 16dc5330051a97aec4ba89d21263cdccc7b7552c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CopyTo<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
516.32 ns 561.22 ns 1.09 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CopyTo&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CopyTo<String>.List(Size: 2048) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 561.2173221749783 > 540.2986903776265. IsChangePoint: Marked as a change because one of 4/29/2023 8:32:21 PM, 5/3/2023 7:05:50 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -6.316839629157004 (T) = (0 -548.5620390608617) / Math.Sqrt((1.8549449480271727 / (7)) + (406.92258441748373 / (14))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (14) - 2, .025) and -0.06651240307956485 = (514.3512981910792 - 548.5620390608617) / 514.3512981910792 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)