search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/arm64: 21 Regressions on 4/8/2023 9:16:19 PM #16000

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 3ab663e128e233ad1f4a1a4a0df64b08a846bcd5
Compare 1411364699b5784040e86f76cb3db8200f6a2c8c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Ctor_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Primitives.StringSegmentBenchmark.Ctor_String.html>) 0.65 ns 2.56 ns 3.96 0.68 True
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Primitives.StringSegmentBenchmark.Trim.html>) 24.70 ns 26.42 ns 1.07 0.01 False
[GetSegmentHashCode - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetSegmentHashCode.html>) 12.59 ns 14.34 ns 1.14 0.02 False
[SubSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Primitives.StringSegmentBenchmark.SubSegment.html>) 1.68 ns 3.28 ns 1.95 0.32 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Ctor_String ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.5611790286007357 > 0.6268890255409992. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -42.810551278379975 (T) = (0 -3.368107700196919) / Math.Sqrt((0.0029117796416218344 / (24)) + (0.0578733428490937 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -4.852327836753944 = (0.5755158962634389 - 3.368107700196919) / 0.5755158962634389 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Trim ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.41756177139642 > 25.94711158869503. IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -24.162130318730785 (T) = (0 -26.451312979737715) / Math.Sqrt((0.06329022153077025 / (24)) + (0.027815605301908946 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.06622826961095019 = (24.808302062174153 - 26.451312979737715) / 24.808302062174153 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetSegmentHashCode ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.340300915825203 > 13.120916195490096. IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -52.718605693693064 (T) = (0 -14.28192058013459) / Math.Sqrt((0.00951698939453071 / (18)) + (0.010436224961463743 / (20))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (20) - 2, .025) and -0.13590098447447313 = (12.573209087183026 - 14.28192058013459) / 12.573209087183026 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### Microsoft.Extensions.Primitives.StringSegmentBenchmark.SubSegment ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.277818709329038 > 1.7691033742768074. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -127.72183791851747 (T) = (0 -3.257107028990416) / Math.Sqrt((0.002216725254208647 / (24)) + (0.0008434791417943145 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.9395862683794332 = (1.6792792783132044 - 3.257107028990416) / 1.6792792783132044 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 3ab663e128e233ad1f4a1a4a0df64b08a846bcd5
Compare 1411364699b5784040e86f76cb3db8200f6a2c8c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Constructors<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MemoryFromArrayStartLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.Constructors(Byte).MemoryFromArrayStartLength.html>) 0.74 ns 2.72 ns 3.69 0.56 True
[MemoryFromArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.Constructors(Byte).MemoryFromArray.html>) 2.10 ns 3.44 ns 1.64 0.37 False
[ArrayAsMemoryStartLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.Constructors(Byte).ArrayAsMemoryStartLength.html>) 1.06 ns 2.74 ns 2.58 0.55 True
[ArrayAsMemory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.Constructors(Byte).ArrayAsMemory.html>) 2.23 ns 3.57 ns 1.60 0.40 False
[ReadOnlyMemoryFromArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.Constructors(Byte).ReadOnlyMemoryFromArray.html>) 2.08 ns 3.55 ns 1.71 0.43 False
[ReadOnlyMemoryFromArrayStartLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.Constructors(Byte).ReadOnlyMemoryFromArrayStartLength.html>) 1.05 ns 2.68 ns 2.54 0.53 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Constructors<Byte>.MemoryFromArrayStartLength ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.724665048217353 > 1.0638981681939756. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -84.22484226497625 (T) = (0 -2.768229012737486) / Math.Sqrt((0.007967010620298521 / (24)) + (0.0012654172584491779 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.668635711392031 = (1.0373199312743606 - 2.768229012737486) / 1.0373199312743606 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Memory.Constructors<Byte>.MemoryFromArray ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.4438833547188006 > 2.276229914187172. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -10.274293047055883 (T) = (0 -3.4149950989764) / Math.Sqrt((0.005412458490944933 / (24)) + (0.2157338359981575 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.6030625724243991 = (2.130294323952506 - 3.4149950989764) / 2.130294323952506 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Memory.Constructors<Byte>.ArrayAsMemoryStartLength ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.7447201550238196 > 1.1091940502036248. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -15.723497895904321 (T) = (0 -2.5962552725871793) / Math.Sqrt((0.0023405511035908175 / (24)) + (0.13012239773358375 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.4208500208463766 = (1.0724560589174696 - 2.5962552725871793) / 1.0724560589174696 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Memory.Constructors<Byte>.ArrayAsMemory ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.573429132394529 > 2.1508153202720695. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -20.978038193359723 (T) = (0 -3.548489230533366) / Math.Sqrt((0.10707750551261 / (24)) + (0.008235811161814396 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.7244562198937595 = (2.057743878677292 - 3.548489230533366) / 2.057743878677292 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Memory.Constructors<Byte>.ReadOnlyMemoryFromArray ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.547611300956706 > 2.2561061568070286. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -27.13979620554341 (T) = (0 -3.5296523011435994) / Math.Sqrt((0.06674440642612658 / (24)) + (0.002480907554502119 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.7188213536754785 = (2.053530632253254 - 3.5296523011435994) / 2.053530632253254 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Memory.Constructors<Byte>.ReadOnlyMemoryFromArrayStartLength ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.675632451364971 > 1.1775987916081738. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -29.578131892963214 (T) = (0 -2.6790280890745772) / Math.Sqrt((0.0017214921310844153 / (24)) + (0.04016268230513221 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -1.4918517300554546 = (1.0751153677249317 - 2.6790280890745772) / 1.0751153677249317 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 3ab663e128e233ad1f4a1a4a0df64b08a846bcd5
Compare 1411364699b5784040e86f76cb3db8200f6a2c8c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SingleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Byte).SingleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20True%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20False).html>) 1.92 μs 2.12 μs 1.11 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1202690000000004 > 1.9994696400000003. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 3/17/2023 1:30:52 PM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -68.55286075664824 (T) = (0 -2116.4629178702257) / Math.Sqrt((226.55376515640089 / (24)) + (4.089148922124619 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.11240192114262644 = (1902.606313099727 - 2116.4629178702257) / 1902.606313099727 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 3ab663e128e233ad1f4a1a4a0df64b08a846bcd5
Compare 1411364699b5784040e86f76cb3db8200f6a2c8c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Order00LinqQueryX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Order00LinqQueryX.html>) 99.77 ms 115.00 ms 1.15 0.01 False
[Order00LinqMethodX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Order00LinqMethodX.html>) 101.34 ms 115.74 ms 1.14 0.01 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'
### Payloads [Baseline]() [Compare]() ### Histogram #### LinqBenchmarks.Order00LinqQueryX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 114.99868666666667 > 105.08449300000001. IsChangePoint: Marked as a change because one of 1/13/2023 3:30:06 PM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -93.60887998151621 (T) = (0 -114722654.08163264) / Math.Sqrt((216670872405.61298 / (24)) + (205844082103.33084 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.1437689558569948 = (100302297.49999999 - 114722654.08163264) / 100302297.49999999 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### LinqBenchmarks.Order00LinqMethodX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 115.74249642857143 > 106.367564625. IsChangePoint: Marked as a change because one of 1/13/2023 3:30:06 PM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -83.263820313454 (T) = (0 -115286878.4458399) / Math.Sqrt((262681097124.546 / (24)) + (247232628991.20227 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.13914689149648718 = (101204576.25476952 - 115286878.4458399) / 101204576.25476952 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 3ab663e128e233ad1f4a1a4a0df64b08a846bcd5
Compare 1411364699b5784040e86f76cb3db8200f6a2c8c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Round - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Decimal.Round.html>) 12.66 ns 16.16 ns 1.28 0.42 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Decimal.Round ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.163623037401393 > 12.631933591452405. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -19.426201761871635 (T) = (0 -16.15975734627917) / Math.Sqrt((0.2541655401408489 / (24)) + (0.4272270611689976 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.3222781528009235 = (12.221148259954738 - 16.15975734627917) / 12.221148259954738 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 3ab663e128e233ad1f4a1a4a0df64b08a846bcd5
Compare 1411364699b5784040e86f76cb3db8200f6a2c8c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.Logging.LoggingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[NoArguments_FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_FilteredByLevel.html>) 23.91 ns 26.16 ns 1.09 0.18 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.LoggingOverhead*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_FilteredByLevel ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.155095612080366 > 25.086275111301642. IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -8.467666449426988 (T) = (0 -25.314099527418456) / Math.Sqrt((0.3167032188042742 / (24)) + (0.1601664275686209 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.055413120228510235 = (23.985015007144913 - 25.314099527418456) / 23.985015007144913 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 3ab663e128e233ad1f4a1a4a0df64b08a846bcd5
Compare 1411364699b5784040e86f76cb3db8200f6a2c8c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EnumerateArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase%3a%20Json400KB).html>) 1.55 μs 1.64 μs 1.06 0.00 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: Json400KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6426093265782942 > 1.6259536552706906. IsChangePoint: Marked as a change because one of 1/30/2023 10:02:08 AM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -87.34413670154903 (T) = (0 -1642.4734732934921) / Math.Sqrt((3.262487702196795 / (24)) + (14.03235584095838 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.06014787819682725 = (1549.2871391556473 - 1642.4734732934921) / 1549.2871391556473 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline fef68da1996ec83e05e2428791909d518f3e32ea
Compare 1411364699b5784040e86f76cb3db8200f6a2c8c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Constructors<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadOnlyMemoryFromArrayStartLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.Constructors(String).ReadOnlyMemoryFromArrayStartLength.html>) 2.74 ns 4.07 ns 1.48 0.39 False
[ReadOnlyMemoryFromArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.Constructors(String).ReadOnlyMemoryFromArray.html>) 2.12 ns 3.51 ns 1.66 0.38 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Constructors<String>.ReadOnlyMemoryFromArrayStartLength ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.065553901252644 > 2.8969362580325755. IsChangePoint: Marked as a change because one of 1/10/2023 5:57:38 PM, 3/14/2023 8:42:09 PM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -20.085989512048535 (T) = (0 -3.9958332728271384) / Math.Sqrt((0.11997863140644119 / (23)) + (0.0080789364873238 / (14))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (14) - 2, .025) and -0.6197243976208507 = (2.4669834440331084 - 3.9958332728271384) / 2.4669834440331084 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Memory.Constructors<String>.ReadOnlyMemoryFromArray ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.5107441854417725 > 2.2935205805562826. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -24.470312959089345 (T) = (0 -3.5475220291422755) / Math.Sqrt((0.05579333475182323 / (24)) + (0.0016471769576866617 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.5172093242991181 = (2.338188918513976 - 3.5475220291422755) / 2.338188918513976 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 3ab663e128e233ad1f4a1a4a0df64b08a846bcd5
Compare 1411364699b5784040e86f76cb3db8200f6a2c8c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in FractalPerf.Launch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/FractalPerf.Launch.Test.html>) 243.20 ms 263.88 ms 1.09 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'FractalPerf.Launch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### FractalPerf.Launch.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 263.8782307692308 > 255.67867096153847. IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -131.79071584517584 (T) = (0 -263148488.14233384) / Math.Sqrt((64444655684.21465 / (24)) + (273192514483.33356 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.08063611250987562 = (243512580.31822345 - 263148488.14233384) / 243512580.31822345 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 3ab663e128e233ad1f4a1a4a0df64b08a846bcd5
Compare 1411364699b5784040e86f76cb3db8200f6a2c8c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse_SequentialAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length%3a%204096%2c%20Chunks%3a%2016).html>) 650.32 ns 712.15 ns 1.10 0.20 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length: 4096, Chunks: 16) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 712.1479832848837 > 683.7641582083195. IsChangePoint: Marked as a change because one of 1/10/2023 5:57:38 PM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -12.90117151200224 (T) = (0 -714.6610032512335) / Math.Sqrt((222.55958878204757 / (24)) + (181.3079273312339 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.09301784697647952 = (653.8420257529539 - 714.6610032512335) / 653.8420257529539 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 3ab663e128e233ad1f4a1a4a0df64b08a846bcd5
Compare 1411364699b5784040e86f76cb3db8200f6a2c8c
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EnumerateProperties - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase%3a%20ObjectProperties).html>) 2.02 μs 2.16 μs 1.07 0.00 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: ObjectProperties) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.158678630615345 > 2.115490022892168. IsChangePoint: Marked as a change because one of 1/30/2023 10:02:08 AM, 3/16/2023 8:14:39 PM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM falls between 4/3/2023 6:15:03 PM and 4/12/2023 8:39:35 PM. IsRegressionStdDev: Marked as regression because -181.8019917130959 (T) = (0 -2157.878317902597) / Math.Sqrt((11.808493128745516 / (24)) + (1.8594705088009176 / (14))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (14) - 2, .025) and -0.07134884684239987 = (2014.1696369604906 - 2157.878317902597) / 2014.1696369604906 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)