search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 35 Improvements on 3/16/2023 9:46:36 PM #14337

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 5bdc36e1d956fc39cd768b6cf59ac4b4bf5f56a5
Compare 7e709b6592b1f5677647edbe3a8093638ea6b0dc
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IterateTryGetSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Char).IterateTryGetSingleSegment.html>) 286.30 ns 232.94 ns 0.81 0.12 False
[IterateGetPositionSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Char).IterateGetPositionSingleSegment.html>) 1.25 μs 855.05 ns 0.68 0.10 False
[IterateGetPositionArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Char).IterateGetPositionArray.html>) 1.20 μs 661.52 ns 0.55 0.10 False
[IterateTryGetTenSegments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Char).IterateTryGetTenSegments.html>) 1.69 μs 1.36 μs 0.80 0.06 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateTryGetSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 232.9352177527562 < 273.843544338431. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/16/2023 8:14:39 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 12.374966607713215 (T) = (0 -243.7644713412592) / Math.Sqrt((5110.0365777220795 / (299)) + (2306.4602227844025 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.3739863585999809 = (389.39162858512714 - 243.7644713412592) / 389.39162858512714 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 855.0465344254512 < 1.1721867757779765. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/16/2023 9:46:36 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 5.101081283645849 (T) = (0 -873.8602897703518) / Math.Sqrt((21198.985406507385 / (299)) + (158915.68488829117 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.3551242234954839 = (1355.0831363321213 - 873.8602897703518) / 1355.0831363321213 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionArray ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 661.5223870681668 < 1.1602091855568148. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/15/2023 8:07:42 AM, 3/15/2023 9:07:40 PM, 3/16/2023 10:20:48 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 10.438989454765823 (T) = (0 -750.608612869286) / Math.Sqrt((140590.1891852308 / (299)) + (68308.05094902722 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.45904231661132927 = (1387.555137709328 - 750.608612869286) / 1387.555137709328 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateTryGetTenSegments ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.3628022132512563 < 1.6109684844905516. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/16/2023 12:15:41 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 27.75987905597681 (T) = (0 -1428.6669368006671) / Math.Sqrt((200874.73258173678 / (299)) + (19851.895726062936 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.4293180992293438 = (2503.4383162868435 - 1428.6669368006671) / 2503.4383162868435 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 5bdc36e1d956fc39cd768b6cf59ac4b4bf5f56a5
Compare 7e709b6592b1f5677647edbe3a8093638ea6b0dc
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).Dictionary(Size%3a%20512).html>) 156.60 μs 130.87 μs 0.84 0.05 False
[ImmutableStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).ImmutableStack(Size%3a%20512).html>) 33.19 μs 11.79 μs 0.36 0.18 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).SortedList(Size%3a%20512).html>) 2.42 ms 1.97 ms 0.81 0.07 False
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).HashSet(Size%3a%20512).html>) 143.00 μs 122.76 μs 0.86 0.06 False
[ImmutableQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).ImmutableQueue(Size%3a%20512).html>) 78.28 μs 40.66 μs 0.52 0.11 False
[ConcurrentStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).ConcurrentStack(Size%3a%20512).html>) 83.27 μs 64.44 μs 0.77 0.11 False
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).ImmutableSortedDictionary(Size%3a%20512).html>) 4.84 ms 3.37 ms 0.70 0.04 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 130.87321409970238 < 147.08561155273478. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/16/2023 2:58:09 AM, 3/16/2023 8:14:39 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 4.565361280288328 (T) = (0 -149398.0088098782) / Math.Sqrt((8462289270.873199 / (299)) + (3096441535.020733 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.29707479869073994 = (212537.56236312378 - 149398.0088098782) / 212537.56236312378 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.ImmutableStack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.792053465408804 < 31.733766248867575. IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 3/16/2023 9:46:36 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 40.70950132151176 (T) = (0 -11852.476200705181) / Math.Sqrt((97929109.62834172 / (299)) + (67863.5366640066 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.6640840187009487 = (35284.04976407908 - 11852.476200705181) / 35284.04976407908 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.9661664663461538 < 2.322348839076236. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/16/2023 9:46:36 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 26.038777921027446 (T) = (0 -1975058.292943392) / Math.Sqrt((96632008484.53569 / (299)) + (2324723229.1865573 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.2189912659042436 = (2528855.57705074 - 1975058.292943392) / 2528855.57705074 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.HashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 122.7580057742782 < 136.1522624349636. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/16/2023 10:20:48 AM, 3/16/2023 9:46:36 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 5.912273545048112 (T) = (0 -135679.28399199605) / Math.Sqrt((4728166343.199466 / (299)) + (1878310240.8034525 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.3232588634829236 = (200489.19250022926 - 135679.28399199605) / 200489.19250022926 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.ImmutableQueue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 40.65824442254174 < 72.24029440955532. IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 3/16/2023 9:46:36 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 7.932618359966723 (T) = (0 -45252.51857239215) / Math.Sqrt((485142218.05597794 / (299)) + (358503745.0210264 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.44859905462241983 = (82068.26439407862 - 45252.51857239215) / 82068.26439407862 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.ConcurrentStack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 64.43972908093278 < 77.95615554468641. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/17/2023 8:17:18 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 22.6063696534228 (T) = (0 -63687.627829258265) / Math.Sqrt((337030939.60753804 / (299)) + (2381775.996433019 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.2862081496065561 = (89224.36953315379 - 63687.627829258265) / 89224.36953315379 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.3738041683333337 < 4.627771404614956. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/16/2023 9:46:36 PM, 3/19/2023 10:05:03 PM, 3/20/2023 4:51:16 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 27.669416663049798 (T) = (0 -3328951.1221474363) / Math.Sqrt((276317506765.4824 / (299)) + (4023591504.3085794 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.3105189482554674 = (4828198.126292937 - 3328951.1221474363) / 4828198.126292937 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 5bdc36e1d956fc39cd768b6cf59ac4b4bf5f56a5
Compare 7e709b6592b1f5677647edbe3a8093638ea6b0dc
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[op_Subtraction - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.op_Subtraction.html>) 263.90 ns 25.20 ns 0.10 0.12 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20null).html>) 7.20 μs 6.13 μs 0.85 0.07 False
[GetNow - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.GetNow.html>) 5.63 μs 4.04 μs 0.72 0.16 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.Parse(value%3a%20%2212%2f30%2f2017%203%3a45%3a22%20AM%20-08%3a00%22).html>) 9.77 μs 7.99 μs 0.82 0.06 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_DateTimeOffset.op_Subtraction ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 25.20409386401971 < 250.47933985817102. IsChangePoint: Marked as a change because one of 3/16/2023 7:54:34 AM, 3/16/2023 8:14:39 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 56.59596786453511 (T) = (0 -29.84816596183377) / Math.Sqrt((3648.1463596968306 / (299)) + (65.17926433775932 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.8823072640614189 = (253.6109448369887 - 29.84816596183377) / 253.6109448369887 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTimeOffset.ToString(format: null) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 6.12715548288749 < 6.846810212094445. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/15/2023 10:57:55 AM, 3/16/2023 2:58:09 AM, 3/16/2023 1:49:47 PM, 3/17/2023 1:34:47 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 22.118742502343242 (T) = (0 -6222.1719239094455) / Math.Sqrt((5433193.4312478425 / (299)) + (12894.902990711573 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.32844991283832875 = (9265.387709511977 - 6222.1719239094455) / 9265.387709511977 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTimeOffset.GetNow ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 4.0446347002051475 < 5.119384378573558. IsChangePoint: Marked as a change because one of 1/30/2023 2:40:17 PM, 3/15/2023 9:07:40 PM, 3/16/2023 7:54:34 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 2.4935416531069494 (T) = (0 -4811.631697897684) / Math.Sqrt((1831656.2169940337 / (299)) + (5386048.815297958 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.20610093970337762 = (6060.760036798541 - 4811.631697897684) / 6060.760036798541 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTimeOffset.Parse(value: "12/30/2017 3:45:22 AM -08:00") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 7.994517943761626 < 9.264946373517425. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/22/2023 4:57:34 AM, 3/16/2023 2:58:09 AM, 3/16/2023 1:49:47 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 35.189021859451685 (T) = (0 -7918.790769420155) / Math.Sqrt((9023584.494173836 / (299)) + (13204.375296983142 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.4383195343429377 = (14098.390906574672 - 7918.790769420155) / 14098.390906574672 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 5bdc36e1d956fc39cd768b6cf59ac4b4bf5f56a5
Compare 7e709b6592b1f5677647edbe3a8093638ea6b0dc
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortSingleLine).html>) 1.20 μs 952.39 ns 0.80 0.15 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20ShortSingleLine).html>) 1.40 μs 1.16 μs 0.83 0.15 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 952.3904351097302 < 1.1240899419998949. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/15/2023 2:53:22 PM, 3/16/2023 8:14:39 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 22.52578032030459 (T) = (0 -961.4639345863758) / Math.Sqrt((91272.86274493368 / (299)) + (593.6100160109742 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.3005893480636882 = (1374.6772828302971 - 961.4639345863758) / 1374.6772828302971 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.1600234258100914 < 1.340086448391945. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/18/2023 5:00:39 PM, 3/19/2023 10:05:03 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 20.77298294949265 (T) = (0 -1164.931930134898) / Math.Sqrt((105623.17096012546 / (299)) + (382.8062510154094 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2699086526552574 = (1595.5975021093184 - 1164.931930134898) / 1595.5975021093184 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 5bdc36e1d956fc39cd768b6cf59ac4b4bf5f56a5
Compare 7e709b6592b1f5677647edbe3a8093638ea6b0dc
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[InterpolateIntoStringBuilder_Flags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value%3a%2032).html>) 10.30 μs 9.00 μs 0.87 0.06 False
[ToString_Format_NonFlags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Enum.ToString_Format_NonFlags(value%3a%20Saturday%2c%20format%3a%20%22D%22).html>) 380.10 ns 343.96 ns 0.90 0.19 False
[Parse_Flags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Enum.Parse_Flags(text%3a%20%22Red%2c%20Orange%2c%20Yellow%2c%20Green%2c%20Blue%22).html>) 3.71 μs 2.35 μs 0.63 0.09 False
[Parse_Flags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Enum.Parse_Flags(text%3a%20%22Red%22).html>) 1.66 μs 1.30 μs 0.78 0.08 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: 32) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.002738497788036 < 9.590147683125052. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 3/16/2023 7:54:34 AM, 3/16/2023 8:14:39 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 25.57098008652622 (T) = (0 -9036.610092705107) / Math.Sqrt((3153781.5195081336 / (299)) + (21048.547570786635 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.23400897068634924 = (11797.279272059048 - 9036.610092705107) / 11797.279272059048 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Enum.ToString_Format_NonFlags(value: Saturday, format: "D") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 343.9621879715279 < 371.1979485272031. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/18/2023 12:49:26 AM, 3/18/2023 5:00:39 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 17.520277574125846 (T) = (0 -340.8518993316235) / Math.Sqrt((7305.714868123202 / (299)) + (29.011753070352775 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.21401168797159253 = (433.6602645553135 - 340.8518993316235) / 433.6602645553135 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Enum.Parse_Flags(text: "Red, Orange, Yellow, Green, Blue") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.346260015530717 < 3.5366000573184424. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/17/2023 8:17:18 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 2.872311614652111 (T) = (0 -3194.1600267838276) / Math.Sqrt((557079.0868271787 / (299)) + (3180999.4891908923 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.2871554259068145 = (4480.864613225317 - 3194.1600267838276) / 4480.864613225317 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Enum.Parse_Flags(text: "Red") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.3042874850312207 < 1.6385785272464295. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/16/2023 8:14:39 PM, 3/17/2023 8:17:18 AM, 3/18/2023 12:49:26 AM, 3/18/2023 5:00:39 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 29.902493518737973 (T) = (0 -1277.2700039830045) / Math.Sqrt((89759.53723180426 / (299)) + (525.2665239390514 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.30929516525132156 = (1849.2269631321678 - 1277.2700039830045) / 1849.2269631321678 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 5bdc36e1d956fc39cd768b6cf59ac4b4bf5f56a5
Compare 7e709b6592b1f5677647edbe3a8093638ea6b0dc
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<StructRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 7.13 μs 5.46 μs 0.77 0.09 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 12.15 μs 9.30 μs 0.77 0.09 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToWriter(Mode%3a%20SourceGen).html>) 5.04 μs 4.14 μs 0.82 0.09 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToWriter(Mode%3a%20Reflection).html>) 10.20 μs 8.05 μs 0.79 0.12 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 5.457689230631277 < 6.497993681990517. IsChangePoint: Marked as a change because one of 2/15/2023 6:58:43 PM, 3/16/2023 8:14:39 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 4.270258594102154 (T) = (0 -5957.096833669521) / Math.Sqrt((1518937.1895582099 / (190)) + (4896011.773608372 / (19))) is greater than 1.9714903918671234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (190) + (19) - 2, .975) and 0.2697991862883808 = (8158.162414787693 - 5957.096833669521) / 8158.162414787693 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.300232533947728 < 11.52502108314463. IsChangePoint: Marked as a change because one of 2/16/2023 6:27:39 AM, 3/18/2023 5:00:39 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 23.904182560224204 (T) = (0 -9597.321736005893) / Math.Sqrt((4958235.8956571 / (201)) + (31487.226017827696 / (8))) is greater than 1.9714903918671234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (201) + (8) - 2, .975) and 0.29639179443266755 = (13640.150384925362 - 9597.321736005893) / 13640.150384925362 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 4.137676859067802 < 4.83482824512867. IsChangePoint: Marked as a change because one of 2/16/2023 3:28:45 AM, 3/15/2023 9:07:40 PM, 3/16/2023 1:49:47 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 3.287969054197193 (T) = (0 -4646.594863596475) / Math.Sqrt((1526629.6406752327 / (189)) + (4034701.7128384374 / (20))) is greater than 1.9714903918671234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (189) + (20) - 2, .975) and 0.24478261171774318 = (6152.658738651612 - 4646.594863596475) / 6152.658738651612 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.049260997666817 < 9.664610095644935. IsChangePoint: Marked as a change because one of 2/16/2023 1:00:08 AM, 3/19/2023 10:05:03 PM, 3/20/2023 4:51:16 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 10.525948968250288 (T) = (0 -8292.309849624462) / Math.Sqrt((6256717.921060004 / (207)) + (118145.48887585866 / (2))) is greater than 1.9714903918671234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (207) + (2) - 2, .975) and 0.27500595654072413 = (11437.762729826145 - 8292.309849624462) / 11437.762729826145 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 5bdc36e1d956fc39cd768b6cf59ac4b4bf5f56a5
Compare 7e709b6592b1f5677647edbe3a8093638ea6b0dc
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).ConcurrentStack(Size%3a%20512).html>) 78.23 μs 60.63 μs 0.78 0.09 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).Queue(Size%3a%20512).html>) 50.78 μs 27.06 μs 0.53 0.06 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).SortedDictionary(Size%3a%20512).html>) 981.00 μs 791.23 μs 0.81 0.06 False
[ImmutableQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).ImmutableQueue(Size%3a%20512).html>) 65.93 μs 39.45 μs 0.60 0.15 False
[ImmutableStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).ImmutableStack(Size%3a%20512).html>) 31.30 μs 9.83 μs 0.31 0.12 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<Int32>.ConcurrentStack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 60.63083275566488 < 75.14585490390557. IsChangePoint: Marked as a change because one of 3/16/2023 2:58:09 AM, 3/16/2023 8:14:39 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 3.0241647137302823 (T) = (0 -67332.53767622825) / Math.Sqrt((611878014.5068325 / (299)) + (899757966.4080327 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.23993931749780778 = (88588.37093712455 - 67332.53767622825) / 88588.37093712455 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 27.060666794427412 < 45.51433500956161. IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 3/16/2023 9:46:36 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 2.5031540448561955 (T) = (0 -34687.861850997884) / Math.Sqrt((98402806.55630307 / (299)) + (571797710.2714022 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.290186574110588 = (48868.98526543538 - 34687.861850997884) / 48868.98526543538 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 791.2250028846153 < 927.5157440973793. IsChangePoint: Marked as a change because one of 3/15/2023 5:33:41 AM, 3/15/2023 2:53:22 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 14.06532235978757 (T) = (0 -824020.0978420464) / Math.Sqrt((17989404477.67521 / (299)) + (6839861437.010923 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.23208179456755904 = (1073057.1199025195 - 824020.0978420464) / 1073057.1199025195 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<Int32>.ImmutableQueue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 39.450203582730566 < 62.88027544853326. IsChangePoint: Marked as a change because one of 2/22/2023 4:57:34 AM, 2/28/2023 3:04:58 AM, 3/16/2023 9:46:36 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 39.40387670383092 (T) = (0 -39668.0781444187) / Math.Sqrt((153608550.06025577 / (299)) + (975831.7654196257 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.4281164514843642 = (69363.90852190102 - 39668.0781444187) / 69363.90852190102 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<Int32>.ImmutableStack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.82644524767767 < 29.980837164302965. IsChangePoint: Marked as a change because one of 3/16/2023 9:46:36 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 69.07832849173266 (T) = (0 -9889.790702569004) / Math.Sqrt((31453363.012958955 / (299)) + (11009.765623749045 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.6943781310175212 = (32359.56489467049 - 9889.790702569004) / 32359.56489467049 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 004a2eb687eb57c58cf706e4edfa87858415add5
Compare 7e709b6592b1f5677647edbe3a8093638ea6b0dc
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndRemove(Int32).SortedSet(Size%3a%20512).html>) 1.58 ms 1.30 ms 0.83 0.05 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndRemove(Int32).SortedList(Size%3a%20512).html>) 1.24 ms 983.91 μs 0.79 0.04 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndRemove(Int32).Queue(Size%3a%20512).html>) 87.95 μs 49.01 μs 0.56 0.07 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndRemove(Int32).List(Size%3a%20512).html>) 284.05 μs 194.35 μs 0.68 0.10 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndRemove<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.3047113091517855 < 1.506433942533482. IsChangePoint: Marked as a change because one of 3/16/2023 2:58:09 AM, 3/16/2023 1:49:47 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 24.15982110068015 (T) = (0 -1312775.1421144973) / Math.Sqrt((97755322907.94128 / (299)) + (315168910.9608807 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.2543448352949451 = (1760566.0153024858 - 1312775.1421144973) / 1760566.0153024858 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndRemove<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 983.9056490384615 < 1.184801892609714. IsChangePoint: Marked as a change because one of 3/9/2023 8:37:21 PM, 3/16/2023 9:46:36 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 5.533498342250359 (T) = (0 -1056496.9891326684) / Math.Sqrt((6489178988.55069 / (299)) + (87119687732.51477 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.26750396937744564 = (1442324.5245912704 - 1056496.9891326684) / 1442324.5245912704 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndRemove<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 49.011713502015226 < 83.7701767249451. IsChangePoint: Marked as a change because one of 3/17/2023 8:17:18 AM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 2.530186830437435 (T) = (0 -64961.964487938065) / Math.Sqrt((721245534.7847662 / (299)) + (1863969237.5987067 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.29810789638535423 = (92552.63615788391 - 64961.964487938065) / 92552.63615788391 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndRemove<Int32>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 194.34711171737212 < 267.2353345756782. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/16/2023 2:58:09 AM, 3/17/2023 1:30:52 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 28.08398149728049 (T) = (0 -194855.90679911242) / Math.Sqrt((4018853192.533543 / (299)) + (14365114.82826226 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.35408662633878735 = (301674.9842082014 - 194855.90679911242) / 301674.9842082014 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 004a2eb687eb57c58cf706e4edfa87858415add5
Compare 7e709b6592b1f5677647edbe3a8093638ea6b0dc
Diff Diff
Configs CompliationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromString(Location).SystemTextJsonSourceGen.html>) 27.47 μs 25.30 μs 0.92 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 25.295632534385113 < 25.97738050555879. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/16/2023 2:58:09 AM, 3/16/2023 1:49:47 PM, 3/21/2023 9:37:45 AM falls between 3/12/2023 3:57:11 PM and 3/21/2023 9:37:45 AM. IsImprovementStdDev: Marked as improvement because 4.198505613162186 (T) = (0 -27354.814809501888) / Math.Sqrt((99251153.45141609 / (299)) + (97105395.75426888 / (19))) is greater than 1.9674995188240387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (19) - 2, .975) and 0.26366199190410294 = (37149.806894036265 - 27354.814809501888) / 37149.806894036265 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
kg commented 1 year ago

Possibly https://github.com/dotnet/runtime/commit/3fec6864a8f728d01f4f4056c025a4aee361ea9e and https://github.com/dotnet/runtime/commit/05c95f131a5aef0d2e435137d022c8274741f5be since many of these cases were expected to improve