search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 31 Regressions on 12/18/2022 12:43:31 PM #11126

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortSingleLine).html>) 105.44 ns 113.09 ns 1.07 0.07 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20LongSingleLine).html>) 137.10 ns 161.53 ns 1.18 0.06 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 113.09490776584732 > 110.60335010866294. IsChangePoint: Marked as a change because one of 11/18/2022 3:21:53 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -11.33823578461305 (T) = (0 -112.02328666039162) / Math.Sqrt((0.11299292718193832 / (17)) + (10.774888097675229 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.06204480117461317 = (105.4788710763376 - 112.02328666039162) / 105.4788710763376 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: LongSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 161.52528595182716 > 143.86687980941153. IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -12.58725537126897 (T) = (0 -155.08368960911895) / Math.Sqrt((30.253540849023626 / (32)) + (5.691510206203128 / (18))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (18) - 2, .025) and -0.10030992742732185 = (140.9454606773623 - 155.08368960911895) / 140.9454606773623 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IterateGetPositionArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Tests.ReadOnlySequenceTests(Char).IterateGetPositionArray.html>) 17.53 ns 24.47 ns 1.40 0.06 False
[FirstSpanSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Tests.ReadOnlySequenceTests(Char).FirstSpanSingleSegment.html>) 9.49 ns 10.80 ns 1.14 0.05 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionArray ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.47229895129983 > 18.527005836317485. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/31/2022 10:23:25 PM, 11/11/2022 5:53:56 PM, 12/3/2022 7:24:45 AM, 12/6/2022 2:12:43 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -6.921993852354634 (T) = (0 -19.963246019244757) / Math.Sqrt((0.8991281731204174 / (17)) + (0.796807828538367 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.10648576953525003 = (18.042026900743412 - 19.963246019244757) / 18.042026900743412 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.795455856190046 > 9.793239253285455. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/25/2022 10:16:52 PM, 12/6/2022 2:12:43 AM, 12/14/2022 12:26:48 AM, 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -14.670108808517359 (T) = (0 -10.928044211849354) / Math.Sqrt((0.18017450096569088 / (32)) + (0.03245673405149954 / (18))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (18) - 2, .025) and -0.13089154372420655 = (9.663211536502924 - 10.928044211849354) / 9.663211536502924 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReplaceWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options%3a%20None).html>) 56.18 μs 62.25 μs 1.11 0.05 False
[Date_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options%3a%20None).html>) 154.82 ns 177.26 ns 1.14 0.05 False
[Date_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options%3a%20Compiled).html>) 74.13 ns 85.53 ns 1.15 0.10 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 62.25053958333333 > 59.05617615406744. IsChangePoint: Marked as a change because one of 11/7/2022 8:57:30 PM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -25.225807058038026 (T) = (0 -62340.28712093568) / Math.Sqrt((181703.8864368521 / (17)) + (1485743.5807044068 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.1055948513179049 = (56386.195220269015 - 62340.28712093568) / 56386.195220269015 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 177.25505544683625 > 162.8360105682268. IsChangePoint: Marked as a change because one of 10/7/2022 9:25:17 PM, 10/26/2022 2:40:44 AM, 12/12/2022 5:25:39 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -214.03909414143553 (T) = (0 -176.56756498423502) / Math.Sqrt((0.035997801438219935 / (17)) + (0.26498011448441755 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.1390964966180801 = (155.00667898501592 - 176.56756498423502) / 155.00667898501592 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 85.52548715266329 > 78.03554793451437. IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 10/21/2022 2:20:53 PM, 11/2/2022 9:35:40 AM, 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -14.365290735534327 (T) = (0 -85.41743568323314) / Math.Sqrt((23.867333366417604 / (32)) + (0.01994294871051982 / (18))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (18) - 2, .025) and -0.17007059998627524 = (73.00195021072666 - 85.41743568323314) / 73.00195021072666 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20JavaScript%2c%26Hello%2b(World)!%2c16).html>) 36.13 ns 40.27 ns 1.11 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,16) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 40.27289005609254 > 38.17028246811943. IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -31.590988389113964 (T) = (0 -40.94819941444359) / Math.Sqrt((0.01610191177557148 / (17)) + (0.6651099697123729 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.12621524330935552 = (36.3591237622733 - 40.94819941444359) / 36.3591237622733 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetByteCount - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Perf_Utf8Encoding.GetByteCount(Input%3a%20EnglishAllAscii).html>) 4.41 μs 5.87 μs 1.33 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishAllAscii) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.8720335365853655 > 4.630046958815641. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/11/2022 5:53:56 PM, 11/23/2022 8:02:34 PM, 12/6/2022 2:12:43 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -13.41264958542208 (T) = (0 -5642.307484790499) / Math.Sqrt((9.469547820012046 / (17)) + (278549.35938923043 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.2794380421903534 = (4409.988837858115 - 5642.307484790499) / 4409.988837858115 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FooObjIsFoo2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/PerfLabTests.CastingPerf.FooObjIsFoo2.html>) 278.49 μs 306.61 μs 1.10 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.CastingPerf.FooObjIsFoo2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 306.6085609243698 > 292.4159021935097. IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/14/2022 12:26:48 AM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -1066.8101350376799 (T) = (0 -306366.40216129186) / Math.Sqrt((5308.801151635796 / (17)) + (12205.782095088545 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.10004562519266695 = (278503.35944714426 - 306366.40216129186) / 278503.35944714426 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Select - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.Select(input%3a%20IEnumerable).html>) 999.40 ns 1.11 μs 1.11 0.06 False
[Select - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.Select(input%3a%20IList).html>) 1.00 μs 1.11 μs 1.11 0.06 False
[SingleWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input%3a%20IEnumerable).html>) 662.27 ns 731.48 ns 1.10 0.37 False
[Range - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.Range.html>) 437.54 ns 466.57 ns 1.07 0.04 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1066340503160286 > 1049.1802179807462. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 12/3/2022 2:55:34 AM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -12.123699225886963 (T) = (0 -1095.6129128526663) / Math.Sqrt((7.284047008576242 / (17)) + (2022.195611675865 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.09520051996378659 = (1000.3765455561447 - 1095.6129128526663) / 1000.3765455561447 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Select(input: IList) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1111679173168518 > 1.055641465769755. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 12/3/2022 7:24:45 AM, 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -13.640417742792174 (T) = (0 -1097.620267192458) / Math.Sqrt((105.0814744560545 / (17)) + (1213.1656907851282 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.08865791785277968 = (1008.2324752272552 - 1097.620267192458) / 1008.2324752272552 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 731.4780999609146 > 705.1720837783336. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 11:06:27 AM, 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -10.112488037536524 (T) = (0 -751.8411303466171) / Math.Sqrt((898.373378610682 / (32)) + (907.9003177570075 / (18))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (18) - 2, .025) and -0.13530557924984035 = (662.2367969365573 - 751.8411303466171) / 662.2367969365573 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Range ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 466.5655367871862 > 460.088794881579. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 8:10:07 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -24.58264504584667 (T) = (0 -467.8258226098738) / Math.Sqrt((0.12150328511243752 / (17)) + (47.621330372951405 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.06755397162229583 = (438.2221742840297 - 467.8258226098738) / 438.2221742840297 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aei%22%2c%20Options%3a%20NonBacktracking).html>) 43.17 μs 55.25 μs 1.28 0.05 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cw%2b%22%2c%20Options%3a%20Compiled).html>) 6.48 ms 7.42 ms 1.15 0.06 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aqj%22%2c%20Options%3a%20Compiled).html>) 23.46 μs 41.91 μs 1.79 0.04 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22zqj%22%2c%20Options%3a%20Compiled).html>) 23.02 μs 31.35 μs 1.36 0.04 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: NonBacktracking) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.246814420803794 > 45.63957957820721. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/10/2022 9:13:35 AM, 11/11/2022 5:53:56 PM, 12/3/2022 7:24:45 AM, 12/6/2022 2:12:43 AM, 12/14/2022 12:26:48 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -230.04332118271427 (T) = (0 -55247.137293627435) / Math.Sqrt((22326.91506744356 / (17)) + (44827.440559225884 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.2742549810340569 = (43356.422471108905 - 55247.137293627435) / 43356.422471108905 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.420949134199133 > 6.8026028618421055. IsChangePoint: Marked as a change because one of 11/10/2022 9:13:35 AM, 11/18/2022 7:31:41 AM, 12/12/2022 5:25:39 PM, 12/14/2022 9:46:59 AM, 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -167.84518776665112 (T) = (0 -7420801.617370084) / Math.Sqrt((640923232.0266442 / (32)) + (186525230.5456634 / (18))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (18) - 2, .025) and -0.14245296629402496 = (6495498.577453249 - 7420801.617370084) / 6495498.577453249 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 41.90969950848973 > 24.632110219594594. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/31/2022 10:23:25 PM, 11/7/2022 4:10:19 PM, 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 12/6/2022 2:12:43 AM, 12/14/2022 12:26:48 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -1420.854602417562 (T) = (0 -41905.42259531243) / Math.Sqrt((2565.896786194888 / (17)) + (586.0746512355241 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.7869360987207377 = (23450.99112683011 - 41905.42259531243) / 23450.99112683011 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.35402986947791 > 24.174443759204713. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/31/2022 10:23:25 PM, 11/7/2022 4:10:19 PM, 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 12/6/2022 2:12:43 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -554.3501684978405 (T) = (0 -31330.512353078273) / Math.Sqrt((3394.7695572291914 / (17)) + (863.0461489249187 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.362216573590415 = (22999.655826018887 - 31330.512353078273) / 22999.655826018887 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value%3a%209223372036854775807).html>) 19.44 ns 24.73 ns 1.27 0.08 False
[TryParseUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value%3a%2018446744073709551615).html>) 20.40 ns 25.13 ns 1.23 0.10 False
[TryParseSingle - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value%3a%20-3.4028235E%2b38).html>) 71.76 ns 78.94 ns 1.10 0.04 False
[TryParseInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value%3a%20-9223372036854775808).html>) 20.65 ns 24.96 ns 1.21 0.06 False
[TryParseSingle - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value%3a%203.4028235E%2b38).html>) 71.91 ns 83.28 ns 1.16 0.05 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.729040197162107 > 20.805457789162592. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -41.16433545021792 (T) = (0 -24.671070779176453) / Math.Sqrt((0.11913617613415182 / (17)) + (0.24306349677955183 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.2500614960359728 = (19.73588568035256 - 24.671070779176453) / 19.73588568035256 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.13267875879347 > 21.091117562234356. IsChangePoint: Marked as a change because one of 10/13/2022 10:51:15 PM, 10/17/2022 3:19:47 PM, 11/7/2022 4:10:19 PM, 12/3/2022 7:24:45 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -30.306758738939962 (T) = (0 -25.331803840979962) / Math.Sqrt((0.12569253070902525 / (17)) + (0.7390885730366072 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.2602324755457629 = (20.100897518935653 - 25.331803840979962) / 20.100897518935653 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.94271918019054 > 75.27215666354125. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/10/2022 12:15:38 AM, 12/21/2022 9:23:17 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -17.735230385168553 (T) = (0 -78.31123709679599) / Math.Sqrt((0.7759207551545092 / (32)) + (1.645456072181125 / (18))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (18) - 2, .025) and -0.08344795187339905 = (72.27964846985715 - 78.31123709679599) / 72.27964846985715 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.959072054830102 > 21.7134524318445. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -28.18657546142394 (T) = (0 -24.854810832654543) / Math.Sqrt((0.18635887557715045 / (17)) + (0.43678781780092424 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.21419693058529166 = (20.470164440848592 - 24.854810832654543) / 20.470164440848592 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 3.4028235E+38) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 83.28081248338322 > 74.48941205021517. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/10/2022 12:29:27 AM, 12/20/2022 7:58:25 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -13.606819083584403 (T) = (0 -78.03928959517668) / Math.Sqrt((1.2682915291766608 / (29)) + (4.178868144185125 / (21))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (21) - 2, .025) and -0.09397458006641075 = (71.33556027457168 - 78.03928959517668) / 71.33556027457168 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ServiceProvider - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider.html>) 27.04 ns 31.39 ns 1.16 0.11 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.39137087728082 > 28.468451239745725. IsChangePoint: Marked as a change because one of 11/5/2022 3:01:26 AM, 11/7/2022 4:10:19 PM, 12/6/2022 11:39:16 PM, 12/12/2022 5:25:39 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -10.571008347009256 (T) = (0 -30.32040383234363) / Math.Sqrt((0.6382190980563519 / (17)) + (1.908068961012693 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.12065407228643853 = (27.055988625001625 - 30.32040383234363) / 27.055988625001625 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in PerfLabTests.DelegatePerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MulticastDelegateCombineInvoke - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/PerfLabTests.DelegatePerf.MulticastDelegateCombineInvoke.html>) 193.52 ms 206.37 ms 1.07 0.11 False
[MulticastDelegateInvoke - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/PerfLabTests.DelegatePerf.MulticastDelegateInvoke(length%3a%201000).html>) 33.53 ms 39.14 ms 1.17 0.09 False
[MulticastDelegateInvoke - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/PerfLabTests.DelegatePerf.MulticastDelegateInvoke(length%3a%20100).html>) 3.42 ms 4.01 ms 1.17 0.08 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.DelegatePerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.DelegatePerf.MulticastDelegateCombineInvoke ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 206.36757333333335 > 202.6833672115385. IsChangePoint: Marked as a change because one of 10/6/2022 10:14:50 PM, 10/7/2022 12:26:39 PM, 10/17/2022 3:19:47 PM, 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -154.65985777785286 (T) = (0 -207032668.0196193) / Math.Sqrt((40197124414.38831 / (17)) + (191681172015.22372 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.07242668465472753 = (193050649.5054945 - 207032668.0196193) / 193050649.5054945 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.DelegatePerf.MulticastDelegateInvoke(length: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 39.13507380952381 > 35.19270035714286. IsChangePoint: Marked as a change because one of 10/5/2022 3:28:28 PM, 10/6/2022 2:13:23 PM, 10/17/2022 3:19:47 PM, 12/6/2022 2:12:43 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -432.6431767412052 (T) = (0 -39118495.744163245) / Math.Sqrt((140625792.18311587 / (17)) + (5266831079.811389 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.1672661635870696 = (33512918.445224244 - 39118495.744163245) / 33512918.445224244 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.DelegatePerf.MulticastDelegateInvoke(length: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.0070182291666665 > 3.594194834134616. IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 12/6/2022 2:12:43 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -330.77799518854556 (T) = (0 -4006752.90047626) / Math.Sqrt((8741183.650151985 / (17)) + (85362380.37059528 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.17010386011546713 = (3424271.1583575746 - 4006752.90047626) / 3424271.1583575746 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in System.Drawing.Tests.Perf_Color

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetBrightness - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Drawing.Tests.Perf_Color.GetBrightness.html>) 760.45 ns 945.06 ns 1.24 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Drawing.Tests.Perf_Color*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Drawing.Tests.Perf_Color.GetBrightness ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 945.058804863053 > 798.8692120261104. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/18/2022 8:48:12 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -84.86688487333343 (T) = (0 -935.2891248283257) / Math.Sqrt((12.327296198476676 / (17)) + (114.01652961025682 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.2277767211014644 = (761.7746034386921 - 935.2891248283257) / 761.7746034386921 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Append_Char_Capacity - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length%3a%20100).html>) 200.02 ns 225.50 ns 1.13 0.26 False
[Append_Char - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Tests.Perf_StringBuilder.Append_Char(length%3a%20100000).html>) 212.80 μs 224.93 μs 1.06 0.10 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 225.4957135555664 > 202.2285117017154. IsChangePoint: Marked as a change because one of 12/15/2022 1:59:35 AM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -5.915170703681398 (T) = (0 -216.8716315983267) / Math.Sqrt((200.53160112599141 / (17)) + (241.5720866599346 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.1353989967802344 = (191.0091802206374 - 216.8716315983267) / 191.0091802206374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Append_Char(length: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 224.93422409456738 > 223.32666459037165. IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 11/25/2022 7:59:25 AM, 12/5/2022 7:07:15 PM, 12/14/2022 12:26:48 AM, 12/21/2022 11:29:15 AM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -10.27912357980589 (T) = (0 -225054.59658872598) / Math.Sqrt((32284796.76481452 / (33)) + (3287806.906054576 / (17))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (17) - 2, .025) and -0.05201183193359046 = (213927.8188298293 - 225054.59658872598) / 213927.8188298293 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 01b57ccc4cc532034f717226f4c7cc07a1ee9518
Compare fc65281072f9259f0b03a291b9c549bc9d792752
Diff Diff

Regressions in GuardedDevirtualization.ThreeClassVirtual

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/GuardedDevirtualization.ThreeClassVirtual.Call(testInput%3a%20pB%3d0.33%20pD%3d0.33).html>) 6.37 ns 7.44 ns 1.17 0.08 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.ThreeClassVirtual*'
### Payloads [Baseline]() [Compare]() ### Histogram #### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.33 pD=0.33) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.443041541890696 > 6.846876648775126. IsChangePoint: Marked as a change because one of 10/6/2022 2:13:23 PM, 10/10/2022 3:36:43 PM, 10/26/2022 7:11:30 AM, 11/2/2022 9:35:40 AM, 11/10/2022 9:13:35 AM, 11/16/2022 6:51:59 PM, 12/1/2022 7:29:59 PM, 12/6/2022 2:12:43 AM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM. IsRegressionStdDev: Marked as regression because -27.439065126562497 (T) = (0 -7.469286441899385) / Math.Sqrt((0.019523658303920492 / (17)) + (0.0003731638205261193 / (33))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (33) - 2, .025) and -0.1429940415666732 = (6.534842851552771 - 7.469286441899385) / 6.534842851552771 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)