dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/arm64: 1327 Improvements on 5/19/2023 1:23:34 PM #18120

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Xml.Linq.Perf_XElement

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateWithElements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Linq.Perf_XElement.CreateWithElements.html>) 85.60 ns 73.48 ns 0.86 0.09 False
[CreateElementsWithNamespace - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Linq.Perf_XElement.CreateElementsWithNamespace.html>) 135.82 ns 112.10 ns 0.83 0.14 True
[GetElement - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Linq.Perf_XElement.GetElement.html>) 43.17 ns 32.19 ns 0.75 0.52 False
[GetElementWithNamespace - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Linq.Perf_XElement.GetElementWithNamespace.html>) 86.10 ns 66.41 ns 0.77 0.01 True
[GetValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Linq.Perf_XElement.GetValue.html>) 73.14 ns 58.27 ns 0.80 0.06 True
[GetAttribute - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Linq.Perf_XElement.GetAttribute.html>) 26.37 ns 17.50 ns 0.66 0.26 False
[CreateElementWithNamespace - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Linq.Perf_XElement.CreateElementWithNamespace.html>) 99.64 ns 73.31 ns 0.74 0.07 True
[CreateElement - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Linq.Perf_XElement.CreateElement.html>) 44.14 ns 35.45 ns 0.80 0.08 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElement*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Xml.Linq.Perf_XElement.CreateWithElements ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 73.48265189408305 < 81.90756544788775. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.50625399548029 (T) = (0 -76.23331582906637) / Math.Sqrt((4.628130644667631 / (299)) + (16.00151194537239 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1333436923038036 = (87.96256965083985 - 76.23331582906637) / 87.96256965083985 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Xml.Linq.Perf_XElement.CreateElementsWithNamespace ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 112.10468885669732 < 130.24785259249313. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.5316561600899 (T) = (0 -113.8245114810113) / Math.Sqrt((7.024494249288065 / (299)) + (5.886866978532024 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17313004219123648 = (137.6570891300133 - 113.8245114810113) / 137.6570891300133 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Xml.Linq.Perf_XElement.GetElement ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 32.19018810974903 < 38.88534699817086. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.116718961736765 (T) = (0 -34.010085729973476) / Math.Sqrt((4.442263860645568 / (299)) + (4.475838231209919 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.199200040245567 = (42.47013915985053 - 34.010085729973476) / 42.47013915985053 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Xml.Linq.Perf_XElement.GetElementWithNamespace ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 66.4098981386651 < 80.77501180996968. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 72.63938758285578 (T) = (0 -67.34218807427318) / Math.Sqrt((2.543097176781413 / (299)) + (0.6910390523131147 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21126422839931655 = (85.37990858156081 - 67.34218807427318) / 85.37990858156081 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Xml.Linq.Perf_XElement.GetValue ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 58.26754603859144 < 69.5062900090972. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 76.24554794148114 (T) = (0 -57.93756389952422) / Math.Sqrt((4.382495403834959 / (299)) + (0.8849269705547137 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27458110440433786 = (79.86773469961798 - 57.93756389952422) / 79.86773469961798 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Xml.Linq.Perf_XElement.GetAttribute ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.49722672468763 < 24.96732908556515. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.52948238407024 (T) = (0 -17.873227337070713) / Math.Sqrt((1.3083687606891434 / (299)) + (0.29282560826327764 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3375596471857711 = (26.980885541076 - 17.873227337070713) / 26.980885541076 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Xml.Linq.Perf_XElement.CreateElementWithNamespace ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 73.30820203492782 < 89.63763780085948. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 78.07042479763238 (T) = (0 -71.27760554214171) / Math.Sqrt((4.181833711151371 / (299)) + (1.102995406302901 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25613813093220866 = (95.8210233728836 - 71.27760554214171) / 95.8210233728836 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Xml.Linq.Perf_XElement.CreateElement ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 35.44756074623873 < 41.816773351823905. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.68028582128147 (T) = (0 -34.29869195401356) / Math.Sqrt((1.097316002809351 / (299)) + (1.006164129303971 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23341882299000644 = (44.74241343596991 - 34.29869195401356) / 44.74241343596991 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Dictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 22.66 μs 18.86 μs 0.83 0.04 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Dictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 22.71 μs 19.24 μs 0.85 0.04 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Dictionary(String%2c%20String)).DeserializeFromStream(Mode%3a%20SourceGen).html>) 24.14 μs 21.83 μs 0.90 0.07 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Dictionary(String%2c%20String)).DeserializeFromReader(Mode%3a%20Reflection).html>) 29.28 μs 25.34 μs 0.87 0.02 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Dictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20Reflection).html>) 23.60 μs 19.77 μs 0.84 0.05 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Dictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20SourceGen).html>) 23.41 μs 19.73 μs 0.84 0.05 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Dictionary(String%2c%20String)).DeserializeFromReader(Mode%3a%20SourceGen).html>) 29.22 μs 24.70 μs 0.85 0.04 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Dictionary(String%2c%20String)).DeserializeFromStream(Mode%3a%20Reflection).html>) 24.53 μs 20.66 μs 0.84 0.06 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Dictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.860656576655053 < 21.509197384312397. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 57.756658195291116 (T) = (0 -18842.992018728084) / Math.Sqrt((83512.56442339806 / (299)) + (46345.76007523728 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15969812463049585 = (22424.07469392151 - 18842.992018728084) / 22424.07469392151 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.238506605691057 < 21.45151230823733. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 74.7924396762388 (T) = (0 -18910.599106973717) / Math.Sqrt((63587.84417471351 / (299)) + (25309.519603038683 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15525938802580758 = (22386.279100254094 - 18910.599106973717) / 22386.279100254094 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 21.834035428436906 < 22.959896276435458. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.82144683176748 (T) = (0 -20832.150717797747) / Math.Sqrt((147034.35037476502 / (299)) + (516481.4476467115 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1393709302661622 = (24205.72514967499 - 20832.150717797747) / 24205.72514967499 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 25.343123651564188 < 27.61333161387411. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.407818221799726 (T) = (0 -24664.471776938997) / Math.Sqrt((110549.48016583627 / (299)) + (249403.8513076395 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1429042842567187 = (28776.799748146845 - 24664.471776938997) / 28776.799748146845 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.7656574571793 < 22.17362787440937. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.383378339658243 (T) = (0 -19845.53040245178) / Math.Sqrt((97870.53640366816 / (299)) + (307960.4845286541 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14308865815338107 = (23159.37417712927 - 19845.53040245178) / 23159.37417712927 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.725437118827468 < 22.301741265287735. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 27.892957829055206 (T) = (0 -19772.415166557297) / Math.Sqrt((97588.66298297905 / (299)) + (187540.63334452317 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14627983658090504 = (23160.30007698311 - 19772.415166557297) / 23160.30007698311 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 24.702334781463932 < 27.44768430661171. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.3849626465617 (T) = (0 -24709.350568910282) / Math.Sqrt((128025.6596885172 / (299)) + (198365.59413886358 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14100754575718333 = (28765.503639599538 - 24709.350568910282) / 28765.503639599538 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.659289223759334 < 23.30241834979773. IsChangePoint: Marked as a change because one of 2/8/2023 4:45:44 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.630665278705056 (T) = (0 -20683.419708374247) / Math.Sqrt((94488.40438412673 / (299)) + (508712.8334286926 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14478766002412077 = (24185.127764828103 - 20683.419708374247) / 24185.127764828103 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ActivityAllocations - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat%3a%20W3C).html>) 330.95 ns 295.35 ns 0.89 0.12 False
[EnumerateActivityEventsSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall.html>) 93.12 ns 36.19 ns 0.39 0.49 False
[EnumerateActivityTagsLarge - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Diagnostics.Perf_Activity.EnumerateActivityTagsLarge.html>) 8.61 μs 6.63 μs 0.77 0.11 False
[EnumerateActivityLinkTagsLarge - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsLarge.html>) 15.93 μs 6.70 μs 0.42 0.38 False
[EnumerateActivityLinksLarge - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Diagnostics.Perf_Activity.EnumerateActivityLinksLarge.html>) 20.00 μs 5.69 μs 0.28 0.29 False
[EnumerateActivityEventsLarge - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Diagnostics.Perf_Activity.EnumerateActivityEventsLarge.html>) 15.13 μs 5.95 μs 0.39 0.45 False
[EnumerateActivityTagObjectsLarge - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge.html>) 10.26 μs 6.76 μs 0.66 0.39 False
[EnumerateActivityLinkTagsSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsSmall.html>) 69.71 ns 60.60 ns 0.87 0.46 False
[EnumerateActivityLinksSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall.html>) 114.75 ns 38.54 ns 0.34 0.33 False
[ActivityAllocations - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat%3a%20Hierarchical).html>) 249.20 ns 226.28 ns 0.91 0.17 False
[EnumerateActivityTagObjectsSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsSmall.html>) 99.07 ns 52.48 ns 0.53 0.44 False

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: W3C) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 295.35387069830824 < 315.8195204958486. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 33.62761394657085 (T) = (0 -300.82971434142326) / Math.Sqrt((82.42987751679287 / (299)) + (32.85445736881438 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1576444019615572 = (357.12912105285756 - 300.82971434142326) / 357.12912105285756 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 36.19364344371358 < 88.88012479520035. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 44.85408492920516 (T) = (0 -37.97615401548891) / Math.Sqrt((504.2141737184837 / (299)) + (7.776856556738847 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6409619686839448 = (105.77195367378542 - 37.97615401548891) / 105.77195367378542 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityTagsLarge ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.626524445656125 < 8.177213079794196. IsChangePoint: Marked as a change because one of 2/20/2023 2:00:07 PM, 2/21/2023 5:46:41 AM, 4/7/2023 7:05:26 PM, 4/8/2023 3:10:58 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.62218887324722 (T) = (0 -6622.005714954726) / Math.Sqrt((713455.1765307385 / (299)) + (129.3966894617154 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29136282413364417 = (9344.705500186164 - 6622.005714954726) / 9344.705500186164 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsLarge ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.7041903022489615 < 13.212915826964938. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.71445882636437 (T) = (0 -6748.398578172147) / Math.Sqrt((10311124.85651792 / (299)) + (1747.3224868756452 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4742345498962287 = (12835.378545395486 - 6748.398578172147) / 12835.378545395486 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityLinksLarge ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.688715506715508 < 19.00476007322343. IsChangePoint: Marked as a change because one of 2/10/2023 11:58:08 PM, 2/11/2023 5:46:38 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.83323383299949 (T) = (0 -5683.585576689782) / Math.Sqrt((34645977.71579011 / (299)) + (597.094382360699 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7412941012784697 = (21969.29256262365 - 5683.585576689782) / 21969.29256262365 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityEventsLarge ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.951354679802956 < 14.40873675796332. IsChangePoint: Marked as a change because one of 2/11/2023 4:30:46 AM, 2/12/2023 12:05:22 AM, 2/12/2023 11:53:55 PM, 2/13/2023 11:10:05 PM, 2/15/2023 9:41:15 PM, 2/16/2023 7:04:07 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.63083623471067 (T) = (0 -5945.9920403378255) / Math.Sqrt((30576165.72942833 / (299)) + (1621.6671598939804 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6694292132449237 = (17987.04628047874 - 5945.9920403378255) / 17987.04628047874 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.759532722594247 < 13.547880004365041. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 30.2374487822406 (T) = (0 -6813.615192428944) / Math.Sqrt((11317243.642929155 / (299)) + (441.586088651303 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4634525092776653 = (12698.99740516169 - 6813.615192428944) / 12698.99740516169 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsSmall ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 60.60462322203033 < 75.38847397923476. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.319070576039874 (T) = (0 -48.45735389878719) / Math.Sqrt((251.92371796187862 / (299)) + (14.18426890692651 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39042230395406896 = (79.49331842865848 - 48.45735389878719) / 79.49331842865848 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 38.542694473802996 < 110.1091783341945. IsChangePoint: Marked as a change because one of 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.16660642142818 (T) = (0 -38.28319844675345) / Math.Sqrt((1000.3506090738932 / (299)) + (0.26148188940767214 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.6933338831532633 = (124.83674049287397 - 38.28319844675345) / 124.83674049287397 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: Hierarchical) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 226.2798280636072 < 238.18132811568333. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 51.98663178160864 (T) = (0 -228.72891708782268) / Math.Sqrt((76.41299307276284 / (299)) + (2.936111179334665 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1362182079359319 = (264.79941947058023 - 228.72891708782268) / 264.79941947058023 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsSmall ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 52.479305793568564 < 80.27393544607662. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 26.973958563258517 (T) = (0 -48.21158446069954) / Math.Sqrt((358.93280303587846 / (299)) + (4.545527853283468 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41058013241794045 = (81.7949769125953 - 48.21158446069954) / 81.7949769125953 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlToStream(CollectionsOfPrimitives).XmlSerializer.html>) 466.83 μs 435.64 μs 0.93 0.13 False
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlToStream(CollectionsOfPrimitives).DataContractSerializer.html>) 703.24 μs 501.39 μs 0.71 0.02 True
[DataContractSerializerBinaryXml - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Xml_ToStream(CollectionsOfPrimitives).DataContractSerializerBinaryXml.html>) 364.88 μs 251.38 μs 0.69 0.36 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.XmlSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 435.6411826086956 < 465.02285864248796. IsChangePoint: Marked as a change because one of 4/2/2023 3:55:31 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.049817441276243 (T) = (0 -419909.7797592879) / Math.Sqrt((166600606.73571074 / (299)) + (213033820.99870315 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09773425447960664 = (465394.7928801171 - 419909.7797592879) / 465394.7928801171 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.DataContractSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 501.3882923076922 < 673.7072398675762. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.323367598097036 (T) = (0 -512561.92517604586) / Math.Sqrt((98827098.37967397 / (299)) + (211007686.54143655 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2731169875537692 = (705150.5075776705 - 512561.92517604586) / 705150.5075776705 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.DataContractSerializer_BinaryXml_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 251.3845999226904 < 357.47642552030675. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 45.93703568111069 (T) = (0 -256812.07024221198) / Math.Sqrt((288611986.99939114 / (214)) + (76131442.57206328 / (13))) is greater than 1.9705633902925863 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (214) + (13) - 2, .975) and 0.3243853050676826 = (380116.17001305637 - 256812.07024221198) / 380116.17001305637 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedDictionaryDeepCopy - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).SortedDictionaryDeepCopy(Size%3a%20512).html>) 13.78 μs 10.68 μs 0.77 0.13 False
[FrozenDictionaryOptimized - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).FrozenDictionaryOptimized(Size%3a%20512).html>) 73.45 μs 48.20 μs 0.66 0.07 True
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).HashSet(Size%3a%20512).html>) 10.62 μs 6.60 μs 0.62 0.04 True
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).ImmutableDictionary(Size%3a%20512).html>) 129.47 μs 90.59 μs 0.70 0.24 False
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).ImmutableList(Size%3a%20512).html>) 11.45 μs 10.79 μs 0.94 0.12 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).SortedDictionary(Size%3a%20512).html>) 66.98 μs 51.40 μs 0.77 0.09 False
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).LinkedList(Size%3a%20512).html>) 12.50 μs 9.79 μs 0.78 0.13 False
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 87.06 μs 69.45 μs 0.80 0.11 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).SortedList(Size%3a%20512).html>) 10.59 μs 9.33 μs 0.88 0.01 False
[ImmutableHashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).ImmutableHashSet(Size%3a%20512).html>) 125.50 μs 82.36 μs 0.66 0.32 False
[ConcurrentStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).ConcurrentStack(Size%3a%20512).html>) 6.75 μs 5.77 μs 0.86 0.15 False
[ConcurrentQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).ConcurrentQueue(Size%3a%20512).html>) 7.67 μs 6.88 μs 0.90 0.01 True
[ImmutableStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).ImmutableStack(Size%3a%20512).html>) 6.59 μs 4.46 μs 0.68 0.17 False
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(Int32).ImmutableSortedSet(Size%3a%20512).html>) 20.27 μs 17.50 μs 0.86 0.09 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<Int32>.SortedDictionaryDeepCopy(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.679951114142792 < 13.195108108176802. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.767703588721524 (T) = (0 -10812.04531095395) / Math.Sqrt((2805230.132502109 / (299)) + (12102.211027273957 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2461693701973106 = (14342.804449036434 - 10812.04531095395) / 14342.804449036434 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.FrozenDictionaryOptimized(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 48.203582519964506 < 71.66710477973288. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 138.07550143225444 (T) = (0 -47842.951213041684) / Math.Sqrt((6346809.548261992 / (173)) + (95369.95332956048 / (13))) is greater than 1.972940542389176 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (173) + (13) - 2, .975) and 0.37715400317799 = (76813.45221315393 - 47842.951213041684) / 76813.45221315393 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.HashSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.598365278110405 < 10.036438355276799. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 135.51840383816742 (T) = (0 -6618.750262829895) / Math.Sqrt((192190.05635007727 / (299)) + (5220.67326411459 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39819980569167795 = (10998.252119936824 - 6618.750262829895) / 10998.252119936824 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 90.58975290697673 < 126.75084774137407. IsChangePoint: Marked as a change because one of 4/8/2023 3:09:32 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 74.77375296370134 (T) = (0 -91892.95102300278) / Math.Sqrt((31470094.262393646 / (299)) + (3120287.212136157 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3234707530263678 = (135829.97547271496 - 91892.95102300278) / 135829.97547271496 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.78889581499951 < 10.873882734005285. IsChangePoint: Marked as a change because one of 1/22/2023 4:26:49 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.942147860345763 (T) = (0 -10828.917380780194) / Math.Sqrt((427721.8947720455 / (299)) + (10734.629489065208 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08779689028464742 = (11871.169112939433 - 10828.917380780194) / 11871.169112939433 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 51.39596721311475 < 63.39178462684502. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 100.15681531981357 (T) = (0 -51204.62403863107) / Math.Sqrt((4489680.426896372 / (299)) + (78530.99128484963 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22108309569946197 = (65738.23697485737 - 51204.62403863107) / 65738.23697485737 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.794323621749198 < 11.905385984187477. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.50625651160189 (T) = (0 -9824.70680850672) / Math.Sqrt((6315772.3697218355 / (299)) + (16041.586857815544 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25513390496002036 = (13189.896645758045 - 9824.70680850672) / 13189.896645758045 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 69.45225396825397 < 81.83548856686879. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 109.73626825209405 (T) = (0 -69676.98795287854) / Math.Sqrt((7363100.802610745 / (299)) + (32830.419919606546 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20604082569778556 = (87758.90525368063 - 69676.98795287854) / 87758.90525368063 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.327209528775754 < 10.114105347675991. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 124.61317529575152 (T) = (0 -9294.732865867001) / Math.Sqrt((39090.558962474875 / (299)) + (589.3767556709273 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15103123775428512 = (10948.262502945721 - 9294.732865867001) / 10948.262502945721 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.ImmutableHashSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 82.35791500443263 < 119.17521170938939. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 62.20744989939745 (T) = (0 -83771.5311309874) / Math.Sqrt((40829649.628396824 / (299)) + (5637328.088890796 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35927615547172104 = (130745.14370955342 - 83771.5311309874) / 130745.14370955342 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.ConcurrentStack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.772279249448124 < 5.942511036092452. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.90047150752456 (T) = (0 -4767.420995613782) / Math.Sqrt((358195.32116941834 / (299)) + (97557.31573978381 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24851595077240804 = (6344.008233460105 - 4767.420995613782) / 6344.008233460105 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.ConcurrentQueue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.880614243159752 < 7.2810375388942195. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 158.43733422339974 (T) = (0 -6896.026685884403) / Math.Sqrt((3649.050342283281 / (299)) + (176.62983954282333 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10448766732561046 = (7700.64959941966 - 6896.026685884403) / 7700.64959941966 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.ImmutableStack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.456260761444144 < 5.680993978009887. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.470042419064356 (T) = (0 -4456.8544157874485) / Math.Sqrt((426957.0144073128 / (299)) + (7330.06958095905 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28327802259172347 = (6218.3867054053335 - 4456.8544157874485) / 6218.3867054053335 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.500292861080485 < 19.193909921362444. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.76531994778877 (T) = (0 -17667.184603501144) / Math.Sqrt((335236.4580520177 / (299)) + (70658.51820616594 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1270845787443823 = (20239.28570088536 - 17667.184603501144) / 20239.28570088536 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ILogB - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Single.ILogB.html>) 11.16 μs 6.38 μs 0.57 0.00 True
[SinCosPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Single.SinCosPi.html>) 90.15 μs 81.21 μs 0.90 0.00 True
[CosPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Single.CosPi.html>) 47.04 μs 44.37 μs 0.94 0.00 True
[SinPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Single.SinPi.html>) 47.02 μs 42.44 μs 0.90 0.00 True
[TanPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Single.TanPi.html>) 50.10 μs 45.34 μs 0.91 0.00 True
[ScaleB - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Single.ScaleB.html>) 15.10 μs 10.04 μs 0.66 0.00 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.MathBenchmarks.Single.ILogB ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.37795809037901 < 10.597371903462088. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 4662.889715267247 (T) = (0 -6382.31917851481) / Math.Sqrt((18.274730955298423 / (299)) + (12.829371352698116 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.42789195614022446 = (11155.793467709265 - 6382.31917851481) / 11155.793467709265 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.MathBenchmarks.Single.SinCosPi ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 81.20793393782385 < 85.67972132621114. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 298.9815865349724 (T) = (0 -81048.37963928212) / Math.Sqrt((12436.171541297126 / (299)) + (11683.542094445715 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10162427579442186 = (90216.5735955879 - 81048.37963928212) / 90216.5735955879 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.MathBenchmarks.Single.CosPi ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 44.366913698907325 < 44.65591425689082. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.2702227994411 (T) = (0 -44202.3309095593) / Math.Sqrt((28713.597655832 / (299)) + (29476.454301435235 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05828114700324845 = (46937.92714130974 - 44202.3309095593) / 46937.92714130974 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.MathBenchmarks.Single.SinPi ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 42.44040542507765 < 44.61957376590188. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 197.3841096044821 (T) = (0 -42542.365449981444) / Math.Sqrt((4006.440147211884 / (299)) + (6285.621149969492 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09373154069966588 = (46942.343643764674 - 42542.365449981444) / 46942.343643764674 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.MathBenchmarks.Single.TanPi ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 45.34342908902691 < 47.59596818167853. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 101.1401866252439 (T) = (0 -45533.7346935646) / Math.Sqrt((5960.724199072466 / (299)) + (27966.11726028507 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09379200390570694 = (50246.44992078254 - 45533.7346935646) / 50246.44992078254 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.MathBenchmarks.Single.ScaleB ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.03864730128028 < 14.376375081967408. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 378.53939694395945 (T) = (0 -10048.545131011002) / Math.Sqrt((47525.4761158173 / (299)) + (68.76666499750367 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3255887023282786 = (14899.72834930512 - 10048.545131011002) / 14899.72834930512 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%2010%2c%20Options%3a%20None).html>) 142.64 ns 132.01 ns 0.93 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%200%2c%20Options%3a%20NonBacktracking).html>) 126.13 ns 101.89 ns 0.81 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%207%2c%20Options%3a%20NonBacktracking).html>) 113.97 ns 101.53 ns 0.89 0.05 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%206%2c%20Options%3a%20NonBacktracking).html>) 132.25 ns 109.79 ns 0.83 0.02 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%204%2c%20Options%3a%20NonBacktracking).html>) 233.00 ns 200.44 ns 0.86 0.02 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%202%2c%20Options%3a%20None).html>) 314.52 ns 266.70 ns 0.85 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%202%2c%20Options%3a%20NonBacktracking).html>) 211.01 ns 180.01 ns 0.85 0.01 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%205%2c%20Options%3a%20NonBacktracking).html>) 251.54 ns 204.22 ns 0.81 0.03 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%206%2c%20Options%3a%20None).html>) 144.37 ns 131.48 ns 0.91 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%204%2c%20Options%3a%20None).html>) 405.33 ns 370.28 ns 0.91 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%208%2c%20Options%3a%20None).html>) 142.83 ns 128.90 ns 0.90 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%208%2c%20Options%3a%20NonBacktracking).html>) 119.30 ns 105.77 ns 0.89 0.04 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%209%2c%20Options%3a%20NonBacktracking).html>) 132.00 ns 110.80 ns 0.84 0.02 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%2012%2c%20Options%3a%20None).html>) 127.66 ns 117.90 ns 0.92 0.01 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%209%2c%20Options%3a%20None).html>) 145.36 ns 130.08 ns 0.89 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%207%2c%20Options%3a%20None).html>) 144.44 ns 132.91 ns 0.92 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%205%2c%20Options%3a%20None).html>) 412.49 ns 372.82 ns 0.90 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%201%2c%20Options%3a%20None).html>) 201.66 ns 174.65 ns 0.87 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%203%2c%20Options%3a%20NonBacktracking).html>) 277.18 ns 242.06 ns 0.87 0.02 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%2011%2c%20Options%3a%20None).html>) 116.51 ns 103.19 ns 0.89 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%203%2c%20Options%3a%20None).html>) 491.06 ns 441.37 ns 0.90 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%2010%2c%20Options%3a%20NonBacktracking).html>) 142.21 ns 119.44 ns 0.84 0.02 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%200%2c%20Options%3a%20None).html>) 86.86 ns 78.71 ns 0.91 0.02 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%2011%2c%20Options%3a%20NonBacktracking).html>) 158.62 ns 145.79 ns 0.92 0.00 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id%3a%2013%2c%20Options%3a%20None).html>) 128.65 ns 116.96 ns 0.91 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 132.01405470790735 < 136.225060799287. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.21808495527769 (T) = (0 -130.4406692318898) / Math.Sqrt((0.565481156552782 / (299)) + (0.37990078738994465 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09232516958387299 = (143.708589090312 - 130.4406692318898) / 143.708589090312 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 101.89397753074248 < 119.01694120516899. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 78.5387807487019 (T) = (0 -104.52538631828784) / Math.Sqrt((5.17018116579259 / (299)) + (1.0751481135537946 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1919865152537246 = (129.3609429688044 - 104.52538631828784) / 129.3609429688044 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 101.52883522924084 < 108.19722227604123. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.82073928392602 (T) = (0 -102.79603425955338) / Math.Sqrt((2.7858715873472186 / (299)) + (0.4837019031437784 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10484793086893507 = (114.83639239010948 - 102.79603425955338) / 114.83639239010948 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 109.79267950171 < 124.95454872610622. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.15151121517212 (T) = (0 -111.8951513014467) / Math.Sqrt((3.093132696209546 / (299)) + (2.116591217619405 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15719026002434291 = (132.76442593636682 - 111.8951513014467) / 132.76442593636682 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 200.44436655933964 < 217.7440901326218. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 54.37243978394624 (T) = (0 -201.64761842437866) / Math.Sqrt((7.940151865989884 / (299)) + (3.167466763689728 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12293257732419628 = (229.91119406667897 - 201.64761842437866) / 229.91119406667897 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 266.70064537941585 < 299.16476772558724. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 126.20635449881487 (T) = (0 -266.8847445280849) / Math.Sqrt((2.922221679564992 / (299)) + (1.7366985742499825 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1518611779695375 = (314.67106279742876 - 266.8847445280849) / 314.67106279742876 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 180.01094462022166 < 198.94167201896732. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.6480749677941 (T) = (0 -180.1748226431473) / Math.Sqrt((9.800864361633938 / (299)) + (14.248835424932027 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1614410148333785 = (214.86243165988688 - 180.1748226431473) / 214.86243165988688 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 204.21686679485626 < 224.68082505244226. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.122759710104454 (T) = (0 -206.2656189108619) / Math.Sqrt((14.062023744855212 / (299)) + (21.912986801308445 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13340472225623795 = (238.01839706291503 - 206.2656189108619) / 238.01839706291503 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 131.47819325542895 < 137.4465822135517. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 138.08449707626306 (T) = (0 -131.9588902647249) / Math.Sqrt((0.4904462536412149 / (299)) + (0.09535071248288822 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09019279928584614 = (145.04049886739043 - 131.9588902647249) / 145.04049886739043 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 370.2827557312693 < 385.86985558188746. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 103.51828095594719 (T) = (0 -368.82963869618015) / Math.Sqrt((3.7802353949477268 / (299)) + (1.6345931736637425 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09453663524818517 = (407.3380028989637 - 368.82963869618015) / 407.3380028989637 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 128.89676305932056 < 135.7932605847718. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 164.29144454625535 (T) = (0 -128.88673386904276) / Math.Sqrt((1.0158848008208365 / (299)) + (0.05245621253092029 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09901430956337195 = (143.05081116946792 - 128.88673386904276) / 143.05081116946792 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 105.77288413006947 < 113.14635651104133. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 45.977859262299006 (T) = (0 -107.35593167425966) / Math.Sqrt((21.86985058051431 / (299)) + (0.6040688635580562 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1290090970081026 = (123.25723644814964 - 107.35593167425966) / 123.25723644814964 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 110.79547553723627 < 124.97614448966156. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 96.11081071050725 (T) = (0 -111.61815311146307) / Math.Sqrt((2.001902236596412 / (299)) + (0.5365968511812393 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1586708919704708 = (132.6688355914407 - 111.61815311146307) / 132.6688355914407 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 117.89921536796537 < 121.72700840473819. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 67.18092413142209 (T) = (0 -116.96769035285939) / Math.Sqrt((1.19661705180274 / (299)) + (0.35696762990929387 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09245593903622436 = (128.8837593500908 - 116.96769035285939) / 128.8837593500908 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 130.08492204766517 < 138.4278586763467. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 114.24804251220144 (T) = (0 -129.53718066712898) / Math.Sqrt((0.654859611180769 / (299)) + (0.248687999601831 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11408768991644164 = (146.21896455520655 - 129.53718066712898) / 146.21896455520655 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 132.91162102655838 < 138.44410692215175. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.3529439295975 (T) = (0 -129.31270314623774) / Math.Sqrt((0.6128604348212017 / (299)) + (1.2928655912505127 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11036480010365461 = (145.3547512073538 - 129.31270314623774) / 145.3547512073538 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 372.8190341323036 < 391.7204911023438. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 115.94084392204371 (T) = (0 -375.3413337903337) / Math.Sqrt((4.600891304455203 / (299)) + (1.2213315448461344 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09267350696619818 = (413.6783579803953 - 375.3413337903337) / 413.6783579803953 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 174.649573444292 < 188.93582425196036. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 126.26747108435667 (T) = (0 -175.03784715661806) / Math.Sqrt((1.7323301444900552 / (299)) + (0.3766225781687727 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11855600879821486 = (198.58079345230615 - 175.03784715661806) / 198.58079345230615 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 242.05595579657594 < 261.09794899817126. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 76.71069055628784 (T) = (0 -243.41273867222407) / Math.Sqrt((13.844439776500002 / (299)) + (1.64497417936627 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11584134733077285 = (275.30436753333146 - 243.41273867222407) / 275.30436753333146 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 103.1870308286751 < 109.33401157103145. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 93.47185270632883 (T) = (0 -103.12024441602179) / Math.Sqrt((0.4611813302188476 / (299)) + (0.2140494473026978 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1084461811711946 = (115.66351042215967 - 103.12024441602179) / 115.66351042215967 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 441.36581753918017 < 465.9863908045995. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 190.07542098003185 (T) = (0 -440.74910612731736) / Math.Sqrt((6.302953044630128 / (299)) + (0.6565635777592326 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10344777018644288 = (491.60449494278043 - 440.74910612731736) / 491.60449494278043 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 119.44088367574402 < 134.70554602737423. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 92.25674341132785 (T) = (0 -120.83896921753166) / Math.Sqrt((2.505003045998022 / (299)) + (0.652388742359006 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15594414724685843 = (143.16466004397586 - 120.83896921753166) / 143.16466004397586 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 78.71276391098773 < 83.04176718442855. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 62.94318552786338 (T) = (0 -77.20418833831981) / Math.Sqrt((0.9612517109149725 / (299)) + (0.32685624449872047 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12071783664263458 = (87.80365570425181 - 77.20418833831981) / 87.80365570425181 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 145.79011068304345 < 149.38361285095118. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.39861427790596 (T) = (0 -144.90678794666553) / Math.Sqrt((3.842918880757914 / (299)) + (3.5393701683517285 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07308752499402758 = (156.33276264377804 - 144.90678794666553) / 156.33276264377804 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 116.96014099356401 < 121.84984196278533. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 71.18684917110691 (T) = (0 -117.06848750061181) / Math.Sqrt((1.115127115154097 / (299)) + (0.30728275742800404 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0913994602229933 = (128.84483595986345 - 117.06848750061181) / 128.84483595986345 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).IDictionary(Size%3a%20512).html>) 7.14 μs 3.94 μs 0.55 0.02 True
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 30.51 μs 24.88 μs 0.82 0.14 False
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).ImmutableDictionary(Size%3a%20512).html>) 18.50 μs 14.37 μs 0.78 0.00 True
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).SortedList(Size%3a%20512).html>) 32.49 μs 25.73 μs 0.79 0.01 True
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).SortedDictionary(Size%3a%20512).html>) 48.25 μs 30.56 μs 0.63 0.08 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyFalse<Int32, Int32>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.9385701105333966 < 6.766898384715825. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 54.128332055977154 (T) = (0 -4063.03996565166) / Math.Sqrt((51190.81648946259 / (299)) + (52715.916192476696 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.46411416211483797 = (7581.913307666757 - 4063.03996565166) / 7581.913307666757 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 24.881240889212826 < 29.02205480286162. IsChangePoint: Marked as a change because one of 2/19/2023 10:06:54 AM, 2/20/2023 3:29:46 AM, 4/19/2023 6:20:25 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.151488228112463 (T) = (0 -26267.50972518841) / Math.Sqrt((11111461.76782623 / (299)) + (2104497.2647240176 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18258869836344283 = (32134.996999182236 - 26267.50972518841) / 32134.996999182236 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.367385598408324 < 17.73796916045484. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 66.94944408041579 (T) = (0 -14393.186009905327) / Math.Sqrt((62442.50796454911 / (299)) + (40718.297147539015 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2118921517636472 = (18262.96495094517 - 14393.186009905327) / 18262.96495094517 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsKeyFalse<Int32, Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 25.73144898378759 < 30.85995971469408. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.02752932441222 (T) = (0 -24150.444566145663) / Math.Sqrt((342814.26599746675 / (299)) + (257232.8995352379 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23059984355951707 = (31388.666045863774 - 24150.444566145663) / 31388.666045863774 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsKeyFalse<Int32, Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 30.564169990967937 < 45.876590747246865. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 121.65923927524521 (T) = (0 -29648.386289539896) / Math.Sqrt((3133716.946260028 / (299)) + (90215.47398589626 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3513197986029861 = (45705.70556290818 - 29648.386289539896) / 45705.70556290818 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Logging.ScopesOverheadBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FilteredByLevel_InsideScope - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel_InsideScope(HasISupportLoggingScopeLogger%3a%20False%2c%20CaptureScopes%3a%20False).html>) 22.45 ns 18.28 ns 0.81 0.37 False
[FilteredByLevel_InsideScope - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel_InsideScope(HasISupportLoggingScopeLogger%3a%20True%2c%20CaptureScopes%3a%20False).html>) 22.83 ns 18.12 ns 0.79 0.35 False
[FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger%3a%20False%2c%20CaptureScopes%3a%20False).html>) 8.62 ns 6.40 ns 0.74 0.10 True
[FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger%3a%20False%2c%20CaptureScopes%3a%20True).html>) 8.29 ns 6.00 ns 0.72 0.07 True
[NotFiltered - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered(HasISupportLoggingScopeLogger%3a%20True%2c%20CaptureScopes%3a%20True).html>) 57.73 ns 52.12 ns 0.90 0.35 False
[FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger%3a%20True%2c%20CaptureScopes%3a%20True).html>) 8.72 ns 5.96 ns 0.68 0.10 True
[FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger%3a%20True%2c%20CaptureScopes%3a%20False).html>) 8.63 ns 5.79 ns 0.67 0.06 True

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.ScopesOverheadBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel_InsideScope(HasISupportLoggingScopeLogger: False, CaptureScopes: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.28244321015296 < 21.285247279212427. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.52472635590368 (T) = (0 -18.137937243453884) / Math.Sqrt((7.138609397350081 / (299)) + (0.044518055011017375 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24965347288745177 = (24.172747641348494 - 18.137937243453884) / 24.172747641348494 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel_InsideScope(HasISupportLoggingScopeLogger: True, CaptureScopes: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.123534347934143 < 21.278736045716425. IsChangePoint: Marked as a change because one of 4/17/2023 2:42:45 PM, 4/20/2023 3:04:43 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 7.793294381827076 (T) = (0 -19.80122216375277) / Math.Sqrt((6.8147409338231055 / (299)) + (3.6026648030853377 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17732164516263124 = (24.069214957852118 - 19.80122216375277) / 24.069214957852118 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger: False, CaptureScopes: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.397253631999259 < 8.133141985468528. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.51353820645302 (T) = (0 -6.098714385936814) / Math.Sqrt((0.043234095486504545 / (299)) + (0.06499593964837798 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2887092926370464 = (8.574151641242791 - 6.098714385936814) / 8.574151641242791 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger: False, CaptureScopes: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.997423525662616 < 8.123492685469026. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.468871021360478 (T) = (0 -5.965794941092592) / Math.Sqrt((0.028853523886142356 / (299)) + (0.2121848831417287 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3053789568411087 = (8.588560625751075 - 5.965794941092592) / 8.588560625751075 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.NotFiltered(HasISupportLoggingScopeLogger: True, CaptureScopes: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 52.120922973623806 < 57.28002009841112. IsChangePoint: Marked as a change because one of 5/20/2023 6:43:26 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.689458213192866 (T) = (0 -52.47197743500813) / Math.Sqrt((14.645557102079207 / (299)) + (1.5854269069834137 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.11310579085490051 = (59.163738914912116 - 52.47197743500813) / 59.163738914912116 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger: True, CaptureScopes: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.961150912796066 < 8.121784187593567. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 35.77320134404109 (T) = (0 -6.026063972519422) / Math.Sqrt((0.038121235808781496 / (299)) + (0.06616985134424477 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3001119189110708 = (8.61003942679478 - 6.026063972519422) / 8.61003942679478 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger: True, CaptureScopes: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.787762171351441 < 8.169433473228482. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 30.408927514239025 (T) = (0 -5.67945204947747) / Math.Sqrt((0.04026656886167554 / (299)) + (0.1209482969999205 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34217791842864664 = (8.633720588872364 - 5.67945204947747) / 8.633720588872364 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToStream(Mode%3a%20Reflection).html>) 869.95 ns 707.79 ns 0.81 0.05 True
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 449.45 ns 396.41 ns 0.88 0.03 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToWriter(Mode%3a%20SourceGen).html>) 356.20 ns 306.13 ns 0.86 0.02 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToString(Mode%3a%20SourceGen).html>) 496.02 ns 440.70 ns 0.89 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 740.57 ns 565.75 ns 0.76 0.02 True
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 819.60 ns 630.62 ns 0.77 0.19 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToString(Mode%3a%20Reflection).html>) 775.85 ns 609.60 ns 0.79 0.02 True
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeObjectProperty(Mode%3a%20Reflection).html>) 1.03 μs 819.29 ns 0.80 0.19 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToStream(Mode%3a%20SourceGen).html>) 522.42 ns 461.54 ns 0.88 0.01 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToWriter(Mode%3a%20Reflection).html>) 665.27 ns 459.97 ns 0.69 0.03 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 707.7916425547963 < 823.5414497637021. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.983177366104616 (T) = (0 -673.017246447489) / Math.Sqrt((300.7538644777104 / (299)) + (654.9450084301062 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23588569251104852 = (880.7808463359003 - 673.017246447489) / 880.7808463359003 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 396.40659019932707 < 426.50412105861426. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/12/2023 6:21:10 AM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.16533038551374 (T) = (0 -398.34856753011695) / Math.Sqrt((303.07585384875375 / (299)) + (2.06927189016932 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11790308410977111 = (451.5927449174857 - 398.34856753011695) / 451.5927449174857 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 306.1257794714441 < 338.393854801955. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/12/2023 8:39:35 PM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 53.62555589131354 (T) = (0 -305.7374820653654) / Math.Sqrt((202.09028631739113 / (299)) + (1.925018443661348 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13734553188905255 = (354.4147667082436 - 305.7374820653654) / 354.4147667082436 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 440.69807306642923 < 469.3009395759125. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/12/2023 8:39:35 PM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 43.94420583108065 (T) = (0 -441.84475660382003) / Math.Sqrt((349.6747004115953 / (299)) + (3.3209663732248287 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10612264614572284 = (494.30132075574545 - 441.84475660382003) / 494.30132075574545 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 565.7474251252677 < 705.2482530476359. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.00270003432543 (T) = (0 -549.1312320205499) / Math.Sqrt((295.05090061687997 / (299)) + (218.4147197246089 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2652470846101586 = (747.3685650218817 - 549.1312320205499) / 747.3685650218817 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 630.623637024473 < 725.2765797741935. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/11/2023 11:54:05 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.75398235253528 (T) = (0 -662.2145169343826) / Math.Sqrt((854.154783831261 / (299)) + (376.95531781252566 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13780837258891918 = (768.0595541420724 - 662.2145169343826) / 768.0595541420724 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 609.5969623106357 < 738.06435942899. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 44.228994714690124 (T) = (0 -602.3931948364433) / Math.Sqrt((317.88688986657576 / (299)) + (212.3437078339173 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23444637939935684 = (786.8726352098154 - 602.3931948364433) / 786.8726352098154 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 819.2903522019615 < 1.0381820553846672. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 25.550796754859235 (T) = (0 -832.6743421379839) / Math.Sqrt((926.0279959722671 / (299)) + (961.0689261349888 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21216806027282614 = (1056.9187413578827 - 832.6743421379839) / 1056.9187413578827 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 461.5383187837757 < 497.6508922059161. IsChangePoint: Marked as a change because one of 3/18/2023 12:49:26 AM, 4/12/2023 8:39:35 PM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.353831754086716 (T) = (0 -470.14985522875264) / Math.Sqrt((288.0958281374501 / (299)) + (48.43735847161802 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11910139440947516 = (533.7161987145842 - 470.14985522875264) / 533.7161987145842 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 459.9722972377961 < 632.103412465292. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 53.19723318821789 (T) = (0 -449.3501662643956) / Math.Sqrt((214.68550663204454 / (299)) + (205.48776934304092 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32489502909695145 = (665.6004408667383 - 449.3501662643956) / 665.6004408667383 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json4KB).html>) 9.37 μs 7.82 μs 0.84 0.01 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json400B).html>) 825.69 ns 620.98 ns 0.75 0.00 True
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfNumbers).html>) 2.82 μs 2.01 μs 0.71 0.00 True
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json40KB).html>) 76.84 μs 61.26 μs 0.80 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json4KB).html>) 6.75 μs 5.50 μs 0.82 0.00 True
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20BroadTree).html>) 14.09 μs 12.25 μs 0.87 0.01 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfStrings).html>) 1.47 μs 1.23 μs 0.84 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20BroadTree).html>) 9.23 μs 8.21 μs 0.89 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfNumbers).html>) 3.26 μs 2.66 μs 0.82 0.00 True
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20DeepTree).html>) 11.63 μs 9.80 μs 0.84 0.00 True
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json40KB).html>) 64.13 μs 51.05 μs 0.80 0.01 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json40KB).html>) 64.11 μs 50.61 μs 0.79 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json4KB).html>) 9.75 μs 8.70 μs 0.89 0.01 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 118.61 ns 81.67 ns 0.69 0.06 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json400B).html>) 931.85 ns 659.83 ns 0.71 0.00 True
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json4KB).html>) 6.48 μs 4.71 μs 0.73 0.00 True
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json4KB).html>) 10.71 μs 8.28 μs 0.77 0.00 True
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json400B).html>) 822.99 ns 623.23 ns 0.76 0.01 True
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 118.76 ns 93.02 ns 0.78 0.03 True
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20DeepTree).html>) 4.19 μs 3.26 μs 0.78 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20DeepTree).html>) 4.42 μs 3.83 μs 0.87 0.01 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json40KB).html>) 108.13 μs 89.40 μs 0.83 0.01 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfStrings).html>) 2.82 μs 2.29 μs 0.81 0.01 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 4.06 μs 3.03 μs 0.75 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json400B).html>) 947.85 ns 774.74 ns 0.82 0.00 True
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 3.64 μs 2.83 μs 0.78 0.01 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json4KB).html>) 7.69 μs 5.44 μs 0.71 0.00 True
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20BroadTree).html>) 8.52 μs 6.81 μs 0.80 0.01 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json40KB).html>) 93.19 μs 79.58 μs 0.85 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 4.04 μs 3.43 μs 0.85 0.01 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfNumbers).html>) 2.81 μs 2.02 μs 0.72 0.00 True
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfStrings).html>) 1.96 μs 1.34 μs 0.68 0.01 True
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20BroadTree).html>) 15.90 μs 13.20 μs 0.83 0.01 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20DeepTree).html>) 4.19 μs 3.28 μs 0.78 0.01 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20DeepTree).html>) 11.61 μs 9.61 μs 0.83 0.00 True
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 92.45 ns 80.08 ns 0.87 0.09 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 3.65 μs 2.89 μs 0.79 0.01 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json4KB).html>) 9.39 μs 7.61 μs 0.81 0.01 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20BroadTree).html>) 14.14 μs 12.61 μs 0.89 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json40KB).html>) 66.04 μs 55.05 μs 0.83 0.01 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfStrings).html>) 1.47 μs 1.24 μs 0.84 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json400B).html>) 1.25 μs 1.10 μs 0.88 0.00 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20BroadTree).html>) 10.43 μs 7.24 μs 0.69 0.01 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json400B).html>) 1.26 μs 966.79 ns 0.76 0.01 True
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20BroadTree).html>) 8.55 μs 6.83 μs 0.80 0.01 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 106.60 ns 92.49 ns 0.87 0.06 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfNumbers).html>) 3.23 μs 2.20 μs 0.68 0.00 True
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 172.54 ns 154.68 ns 0.90 0.10 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json4KB).html>) 6.49 μs 4.74 μs 0.73 0.00 True
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json40KB).html>) 95.90 μs 84.64 μs 0.88 0.01 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20DeepTree).html>) 4.80 μs 3.53 μs 0.74 0.00 True
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json400B).html>) 1.12 μs 937.00 ns 0.84 0.02 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20DeepTree).html>) 12.42 μs 9.97 μs 0.80 0.00 True
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json400B).html>) 1.10 μs 903.08 ns 0.82 0.01 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json40KB).html>) 93.85 μs 79.58 μs 0.85 0.01 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Reader*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.8237155424655445 < 8.93179183688661. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 259.43593339628353 (T) = (0 -7830.1776622062425) / Math.Sqrt((5758.180181163578 / (299)) + (264.20014951396035 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17249407794385013 = (9462.382628936559 - 7830.1776622062425) / 9462.382628936559 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 620.9781572069135 < 785.8890599258665. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 59.575465551509126 (T) = (0 -643.2931275481861) / Math.Sqrt((38.88758329100063 / (299)) + (126.10548567141369 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22502613826967485 = (830.083644513986 - 643.2931275481861) / 830.083644513986 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.005908861724069 < 2.7013970900573603. IsChangePoint: Marked as a change because one of 4/18/2023 4:36:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 66.89381232872594 (T) = (0 -2087.8807410451286) / Math.Sqrt((1492.2589553979894 / (299)) + (1332.60910629267 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24935475939804286 = (2781.4480504409994 - 2087.8807410451286) / 2781.4480504409994 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 61.25887357271635 < 73.32855586221557. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 121.82095964537868 (T) = (0 -61373.89524183247) / Math.Sqrt((408310.56049375463 / (299)) + (200995.83959041498 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20475709909630865 = (77176.28811535309 - 61373.89524183247) / 77176.28811535309 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.503134910147992 < 6.390201074183621. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 77.53191228203019 (T) = (0 -5528.327077862904) / Math.Sqrt((800.5317855839647 / (299)) + (3063.6012402092715 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17797848476988337 = (6725.282702990206 - 5528.327077862904) / 6725.282702990206 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: BroadTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.250668067226893 < 13.456310744266638. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.333736474106907 (T) = (0 -12577.376956598991) / Math.Sqrt((7746.729064599542 / (299)) + (35825.23745014349 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10953412540903462 = (14124.490691320874 - 12577.376956598991) / 14124.490691320874 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2252885565546305 < 1.3945371084250862. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.08295637895892 (T) = (0 -1240.891115959945) / Math.Sqrt((492.2693499996184 / (299)) + (318.0925430482636 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13863869530111791 = (1440.6162770380547 - 1240.891115959945) / 1440.6162770380547 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: BroadTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.205772775144812 < 8.74518895484215. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.587397687699905 (T) = (0 -8269.574893405928) / Math.Sqrt((5489.088012856343 / (299)) + (6767.092795256729 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09544429920541857 = (9142.140043052908 - 8269.574893405928) / 9142.140043052908 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.658386145289061 < 3.101786394954394. IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 139.65928154668288 (T) = (0 -2635.1435615648275) / Math.Sqrt((1086.045992481501 / (299)) + (192.14118171084866 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18527999660952194 = (3234.416180526574 - 2635.1435615648275) / 3234.416180526574 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: DeepTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.803652573796713 < 11.077225559767376. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 161.29615114904522 (T) = (0 -9841.735487986673) / Math.Sqrt((18233.215510937735 / (299)) + (650.3556737035748 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14724881624034816 = (11541.157227828098 - 9841.735487986673) / 11541.157227828098 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 51.04945202225842 < 60.999648350235155. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 163.09398222840798 (T) = (0 -50845.64349517165) / Math.Sqrt((192356.46311409265 / (299)) + (76298.52748769647 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20562727454673319 = (64007.287594320755 - 50845.64349517165) / 64007.287594320755 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 50.61171405455385 < 61.05359642034568. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 95.31358154674999 (T) = (0 -50925.4973749005) / Math.Sqrt((216221.65466440067 / (299)) + (235485.061368536 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2043789186035299 = (64007.22475266282 - 50925.4973749005) / 64007.22475266282 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.695963357732973 < 9.33601900702267. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.130333943251422 (T) = (0 -8901.598598882461) / Math.Sqrt((3152.9799252514435 / (299)) + (22053.721878370492 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09314755707056083 = (9815.928344557684 - 8901.598598882461) / 9815.928344557684 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 81.67441066272912 < 98.59681767380069. IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 46.82667108709498 (T) = (0 -81.88389037418405) / Math.Sqrt((16.297166344026095 / (299)) + (3.2051098457157443 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2388336454050541 = (107.57686526719708 - 81.88389037418405) / 107.57686526719708 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 659.8326492616258 < 889.0799819209516. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 120.92669347617147 (T) = (0 -659.7787834902119) / Math.Sqrt((110.90730133451912 / (299)) + (65.45342679443401 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29880742523564136 = (940.9380635725298 - 659.7787834902119) / 940.9380635725298 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.705790906568635 < 6.187927943454438. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 163.79538722072905 (T) = (0 -4728.064822915308) / Math.Sqrt((2352.2260337357693 / (299)) + (1356.173839529655 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.268437399412716 = (6462.966831710247 - 4728.064822915308) / 6462.966831710247 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.27579308712121 < 10.220793046774576. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 247.30673199022806 (T) = (0 -8231.623345933791) / Math.Sqrt((10180.862429225648 / (299)) + (880.1535009282302 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2325744773369217 = (10726.283010980478 - 8231.623345933791) / 10726.283010980478 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 623.231767330215 < 781.1223654664392. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 131.15384523851168 (T) = (0 -610.3035029822372) / Math.Sqrt((73.9655835035242 / (299)) + (28.771224603684526 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25210943982567274 = (816.0331677939355 - 610.3035029822372) / 816.0331677939355 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 93.0185319984089 < 113.50058641711159. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.403266814231625 (T) = (0 -94.3876999646421) / Math.Sqrt((4.900790024221955 / (299)) + (17.751498277697674 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2256901782412544 = (121.89913819025635 - 94.3876999646421) / 121.89913819025635 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: DeepTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.264361503539321 < 3.9974061095380358. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 149.4273776826965 (T) = (0 -3238.492112013056) / Math.Sqrt((1031.0364978994367 / (299)) + (463.2208446693265 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2238727290952996 = (4172.630228851611 - 3238.492112013056) / 4172.630228851611 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: DeepTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.8348692286632757 < 4.190608221519204. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 44.35538969179794 (T) = (0 -3855.164885780817) / Math.Sqrt((438.7652407158339 / (299)) + (1947.759351918114 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12397455844668497 = (4400.745347013065 - 3855.164885780817) / 4400.745347013065 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 89.40310105363986 < 103.30178870829342. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 183.75852009363902 (T) = (0 -89526.02343403877) / Math.Sqrt((654990.3672171275 / (299)) + (104492.56225548802 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1719035782046832 = (108110.6270691835 - 89526.02343403877) / 108110.6270691835 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.292746749191889 < 2.6767981985112397. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 145.09624241211088 (T) = (0 -2298.838292750035) / Math.Sqrt((1034.2672602190848 / (299)) + (93.1997480479043 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17065364515527714 = (2771.867603108285 - 2298.838292750035) / 2771.867603108285 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.031266749750056 < 3.8581792811575797. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 195.7062225815616 (T) = (0 -3037.9370969180413) / Math.Sqrt((3184.2180417675086 / (299)) + (201.9106838404508 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2479082756088921 = (4039.3172779258402 - 3037.9370969180413) / 4039.3172779258402 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 774.7393419628263 < 900.9776390244552. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 232.19719484578226 (T) = (0 -775.8226025090872) / Math.Sqrt((56.10205057277426 / (299)) + (4.802996291274999 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1825974079974118 = (949.1315663782856 - 775.8226025090872) / 949.1315663782856 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.828330200784051 < 3.4589058077260826. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 30.111035484846358 (T) = (0 -2920.4230490022) / Math.Sqrt((1798.5351027492372 / (299)) + (6620.390930303246 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18965696196364715 = (3603.9342746487405 - 2920.4230490022) / 3603.9342746487405 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.438488378334854 < 7.332938505683043. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 171.5096751239722 (T) = (0 -5471.276080562695) / Math.Sqrt((4882.632779636113 / (299)) + (1965.5797431906628 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2886291416205103 = (7691.17263676828 - 5471.276080562695) / 7691.17263676828 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: BroadTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.811336563588849 < 8.141100776536161. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.725021455768946 (T) = (0 -7154.418825145924) / Math.Sqrt((5375.247859519973 / (299)) + (38955.15725290856 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1594842596970274 = (8511.939136995863 - 7154.418825145924) / 8511.939136995863 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 79.5804373291683 < 89.67062876506023. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 174.85038430651366 (T) = (0 -79813.3888341777) / Math.Sqrt((663468.7799208672 / (299)) + (61138.17433307862 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15416596912345665 = (94360.57893233093 - 79813.3888341777) / 94360.57893233093 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.4256118778245037 < 3.8557302700039466. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.963003056323602 (T) = (0 -3632.318371599992) / Math.Sqrt((1178.3525669508374 / (299)) + (17202.63934418569 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09906303814121008 = (4031.712012465208 - 3632.318371599992) / 4031.712012465208 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.015396163334199 < 2.6894588537433766. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 221.85298210250363 (T) = (0 -2028.7343530012997) / Math.Sqrt((1235.0699791606526 / (299)) + (93.09720473336567 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2687241079338972 = (2774.2393466157296 - 2028.7343530012997) / 2774.2393466157296 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.3390732430022152 < 1.8717092669972457. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 83.24054414628236 (T) = (0 -1356.4130461632994) / Math.Sqrt((348.15278915934744 / (299)) + (652.6247169274994 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3054722006586301 = (1953.000365787524 - 1356.4130461632994) / 1953.000365787524 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: BroadTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.19696851671535 < 15.151127703418881. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.944727240197626 (T) = (0 -13891.961587043696) / Math.Sqrt((17760.093806636665 / (299)) + (164164.53993547228 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12701807777580798 = (15913.22939614788 - 13891.961587043696) / 15913.22939614788 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: DeepTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.284784148132606 < 3.99434413879783. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.69935636556913 (T) = (0 -3411.6178630524273) / Math.Sqrt((1001.2608563361632 / (299)) + (5446.81890629711 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18506522663233763 = (4186.3692341341 - 3411.6178630524273) / 4186.3692341341 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: DeepTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.613701032763533 < 11.087404707282113. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 51.93509064622877 (T) = (0 -9820.825795153594) / Math.Sqrt((21742.35460479138 / (299)) + (13000.924261838869 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14763677340100606 = (11521.87880551766 - 9820.825795153594) / 11521.87880551766 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 80.08039944365198 < 92.13913154062534. IsChangePoint: Marked as a change because one of 1/28/2023 4:01:39 AM, 3/8/2023 10:34:21 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 60.74198387621461 (T) = (0 -79.42288968573675) / Math.Sqrt((8.235831555596445 / (299)) + (0.38604838688270143 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1546747077590445 = (93.95541623413023 - 79.42288968573675) / 93.95541623413023 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.8930033567054583 < 3.466035870201378. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 203.7210766655589 (T) = (0 -2902.9007429797716) / Math.Sqrt((1810.7393733126505 / (299)) + (77.50574663994827 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19568060612627686 = (3609.139310938364 - 2902.9007429797716) / 3609.139310938364 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.60640141413762 < 8.991077801332848. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 44.346378826040315 (T) = (0 -7844.8491648018) / Math.Sqrt((7448.660314915506 / (299)) + (16690.95678830442 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1697871803562047 = (9449.20263718362 - 7844.8491648018) / 9449.20263718362 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: BroadTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.606880712365589 < 13.449630388149291. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 165.4008813390266 (T) = (0 -12597.251397477366) / Math.Sqrt((9988.213492040484 / (299)) + (734.387651423447 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11070786596879133 = (14165.48164029446 - 12597.251397477366) / 14165.48164029446 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 55.05444680986273 < 62.738931202072926. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 104.45303786788692 (T) = (0 -55274.30574340153) / Math.Sqrt((69713.27043193529 / (299)) + (127503.84955238485 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1592120201906531 = (65741.07512328526 - 55274.30574340153) / 65741.07512328526 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2386399757554292 < 1.4042706066783353. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.49678964589774 (T) = (0 -1277.666117601612) / Math.Sqrt((520.2227735871884 / (299)) + (151.7754021477748 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12425932727084621 = (1458.9548680203466 - 1277.666117601612) / 1458.9548680203466 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1046276255673146 < 1.1926503954429215. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.920848659172393 (T) = (0 -1130.3620460240065) / Math.Sqrt((69.85368343051721 / (299)) + (327.18615299710814 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1000041303923267 = (1255.9635929404371 - 1130.3620460240065) / 1255.9635929404371 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: BroadTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.235357542942496 < 9.79165751141392. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.19055184804088 (T) = (0 -7878.261972375397) / Math.Sqrt((10240.194225345991 / (299)) + (134214.94411354134 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23052706336339507 = (10238.517298362143 - 7878.261972375397) / 10238.517298362143 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 966.7882842579683 < 1.189391048162786. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 95.28566492666639 (T) = (0 -944.8651583653728) / Math.Sqrt((325.22318325436885 / (299)) + (129.07073217414708 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.250775896274751 = (1261.1248806163185 - 944.8651583653728) / 1261.1248806163185 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: BroadTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.829011234729493 < 8.171589201081623. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 106.61449511313899 (T) = (0 -6852.19953607485) / Math.Sqrt((5950.879429565269 / (299)) + (2824.483075316559 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19329885950469344 = (8494.09922969449 - 6852.19953607485) / 8494.09922969449 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 92.48596053006614 < 102.21591617379961. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 3/8/2023 10:34:21 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.956335240185854 (T) = (0 -97.99139042857335) / Math.Sqrt((11.52137789877433 / (299)) + (11.496413923547761 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08892737216428886 = (107.5560690055586 - 97.99139042857335) / 107.5560690055586 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.1959650065281253 < 3.067066041973466. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 280.8496271147563 (T) = (0 -2191.04111681833) / Math.Sqrt((1030.3058620046077 / (299)) + (120.65462617951675 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3137918230536002 = (3192.968534080109 - 2191.04111681833) / 3192.968534080109 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 154.68355559018022 < 163.8274473421206. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.01183510856089 (T) = (0 -155.062856647163) / Math.Sqrt((11.226339655712879 / (299)) + (0.9800130139377422 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10652226881416221 = (173.54977212622984 - 155.062856647163) / 173.54977212622984 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.7382731693859 < 6.193741819288362. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 62.808012418048136 (T) = (0 -4929.267278394904) / Math.Sqrt((2325.339830834098 / (299)) + (7744.748185550761 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23840070615599973 = (6472.258204856706 - 4929.267278394904) / 6472.258204856706 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 84.63557952182951 < 91.91656999592895. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 195.73904129145185 (T) = (0 -84529.65269014062) / Math.Sqrt((318630.23771269096 / (299)) + (39175.80787849322 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12884047526579803 = (97031.19840873154 - 84529.65269014062) / 97031.19840873154 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: DeepTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.5286229387847303 < 4.594671758498412. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 251.44536318788053 (T) = (0 -3550.066612692065) / Math.Sqrt((1801.1071492989252 / (299)) + (256.7042851778842 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2644657214885249 = (4826.514163114806 - 3550.066612692065) / 4826.514163114806 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 936.9962110475504 < 1.0653130038383913. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 170.78666508339828 (T) = (0 -940.0074103580675) / Math.Sqrt((74.15788308194017 / (299)) + (11.468309239581702 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16188400633575586 = (1121.5719750775236 - 940.0074103580675) / 1121.5719750775236 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: DeepTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.968933121019107 < 11.770212323384081. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 235.84219070184312 (T) = (0 -9985.815681623228) / Math.Sqrt((16382.75208640678 / (299)) + (491.5606926776202 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18518723936828227 = (12255.35014189187 - 9985.815681623228) / 12255.35014189187 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 903.0839952519359 < 1.0532155355917332. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.82762125276345 (T) = (0 -924.6908761254278) / Math.Sqrt((63.44709979762313 / (299)) + (176.00066177300084 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16374570553499426 = (1105.7532167496963 - 924.6908761254278) / 1105.7532167496963 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 79.5769548028114 < 89.1114082712286. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 184.23168382767824 (T) = (0 -79828.29345283484) / Math.Sqrt((623823.6156328485 / (299)) + (53764.08183863401 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1540073192160842 = (94360.50129755751 - 79828.29345283484) / 94360.50129755751 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Next_int_int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Random.Next_int_int.html>) 5.95 ns 4.11 ns 0.69 0.13 False
[Next_long - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Random.Next_long.html>) 34.36 ns 31.30 ns 0.91 0.10 False
[NextSingle_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Random.NextSingle_unseeded.html>) 4.37 ns 2.55 ns 0.58 0.35 False
[NextDouble - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Random.NextDouble.html>) 5.19 ns 3.29 ns 0.63 0.19 False
[Next_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Random.Next_unseeded.html>) 3.96 ns 2.34 ns 0.59 0.39 False
[Next - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Random.Next.html>) 5.39 ns 3.09 ns 0.57 0.28 False
[Next_int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Random.Next_int.html>) 6.07 ns 3.87 ns 0.64 0.18 False
[NextDouble_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Random.NextDouble_unseeded.html>) 4.08 ns 2.32 ns 0.57 0.33 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Random.Next_int_int ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.114132978633471 < 5.737749897279757. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.45134494278136 (T) = (0 -4.117764682671188) / Math.Sqrt((0.3839203453819387 / (299)) + (0.007110603530780719 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3697283864114929 = (6.533317690172228 - 4.117764682671188) / 6.533317690172228 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Random.Next_long ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 31.297397683371173 < 32.65431581650211. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.68057339033792 (T) = (0 -31.346343024632272) / Math.Sqrt((0.7058828212075555 / (299)) + (0.07022256181293128 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09346245191591636 = (34.57809672735676 - 31.346343024632272) / 34.57809672735676 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Random.NextSingle_unseeded ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.5525313754663626 < 4.054542923285443. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 33.80367693126544 (T) = (0 -2.367638517036169) / Math.Sqrt((0.6758180280229615 / (299)) + (0.018006178424421603 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.46294936488141164 = (4.40859457602794 - 2.367638517036169) / 4.40859457602794 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Random.NextDouble ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.285215909305663 < 4.934199147191837. IsChangePoint: Marked as a change because one of 4/18/2023 4:36:42 PM, 4/19/2023 10:34:24 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 6.851168329829856 (T) = (0 -3.61878690593511) / Math.Sqrt((0.3855399794391819 / (299)) + (0.8079047377507047 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32287690295316146 = (5.344356028789826 - 3.61878690593511) / 5.344356028789826 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Random.Next_unseeded ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.3419932789547397 < 3.7696358017890863. IsChangePoint: Marked as a change because one of 2/26/2023 5:38:06 PM, 3/1/2023 11:13:23 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.571046484369118 (T) = (0 -2.592881631813353) / Math.Sqrt((1.3986261711323513 / (299)) + (0.43957579234994276 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.42002029057959267 = (4.470641971948474 - 2.592881631813353) / 4.470641971948474 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Random.Next ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.0892673300635174 < 5.0860557916591995. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.38205735055621 (T) = (0 -3.1213028914963328) / Math.Sqrt((0.6518797632543483 / (299)) + (0.021453200391259464 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4321591106539674 = (5.4967913548653655 - 3.1213028914963328) / 5.4967913548653655 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Random.Next_int ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.8734624563640545 < 5.703312332156542. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 52.0106729405522 (T) = (0 -3.9230972853727546) / Math.Sqrt((0.2350635129338288 / (299)) + (0.015103077718482835 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3691363302676549 = (6.218613424097154 - 3.9230972853727546) / 6.218613424097154 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Random.NextDouble_unseeded ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.318652976024223 < 4.126152072392254. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.879860178306343 (T) = (0 -2.1836724707642694) / Math.Sqrt((1.2488243592297343 / (299)) + (0.24693019700987226 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5091440178605763 = (4.448702980549628 - 2.1836724707642694) / 4.448702980549628 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_ToString(CollectionsOfPrimitives).SystemTextJsonSourceGen.html>) 148.30 μs 139.07 μs 0.94 0.01 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_ToString(CollectionsOfPrimitives).SystemTextJsonReflection.html>) 153.01 μs 139.94 μs 0.91 0.00 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonToString(CollectionsOfPrimitives).JsonNet.html>) 475.72 μs 360.56 μs 0.76 0.06 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 139.06553413400758 < 142.60702905601343. IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.66394756435591 (T) = (0 -138543.8006308608) / Math.Sqrt((9371655.923831344 / (299)) + (187435.46930117536 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08045321766947178 = (150665.30957754105 - 138543.8006308608) / 150665.30957754105 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 139.94248961945124 < 147.07143673974534. IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.4921810926754 (T) = (0 -140245.2927045752) / Math.Sqrt((8193457.104864277 / (299)) + (275693.06204469653 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0920538171336628 = (154464.32327280496 - 140245.2927045752) / 154464.32327280496 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.JsonNet_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 360.5631766381766 < 454.50271085448173. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 74.14847938085963 (T) = (0 -363659.70998020616) / Math.Sqrt((103620106.94742075 / (299)) + (23431372.978603482 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2301162442305858 = (472356.6476821793 - 363659.70998020616) / 472356.6476821793 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(String).ImmutableSortedDictionary(Size%3a%20512).html>) 466.03 μs 417.58 μs 0.90 0.17 False
[ConcurrentQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(String).ConcurrentQueue(Size%3a%20512).html>) 9.64 μs 9.01 μs 0.94 0.00 True
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(String).ImmutableSortedSet(Size%3a%20512).html>) 445.38 μs 396.82 μs 0.89 0.11 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<String>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 417.5793820488722 < 445.88437106092437. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.18931163519388 (T) = (0 -415648.866834757) / Math.Sqrt((188975572.34379515 / (299)) + (35065218.16114674 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15312824199398894 = (490804.9688814947 - 415648.866834757) / 490804.9688814947 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndClear<String>.ConcurrentQueue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.013204141950899 < 9.167581710192927. IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 126.45362704520964 (T) = (0 -9003.772269454767) / Math.Sqrt((3655.344898096908 / (299)) + (229.37741592691108 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07128593971083093 = (9694.880969769432 - 9003.772269454767) / 9694.880969769432 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndClear<String>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 396.82360491071427 < 426.59542793367336. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.45197159318658 (T) = (0 -402911.1606582794) / Math.Sqrt((111394170.60039 / (299)) + (47548001.97801938 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13547893252985452 = (466051.29223434813 - 402911.1606582794) / 466051.29223434813 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SkipHalfTakeHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SkipHalfTakeHalf(input%3a%20IEnumerable).html>) 756.27 ns 486.76 ns 0.64 0.10 False
[SelectToArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20Array).html>) 265.95 ns 158.00 ns 0.59 0.33 False
[ToDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.ToDictionary(input%3a%20IEnumerable).html>) 2.81 μs 2.26 μs 0.80 0.03 True
[AnyWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input%3a%20IEnumerable).html>) 856.41 ns 256.73 ns 0.30 0.09 True
[SelectToArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20List).html>) 274.19 ns 162.52 ns 0.59 0.26 False
[SingleWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input%3a%20IEnumerable).html>) 867.84 ns 242.70 ns 0.28 0.08 True
[Where - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Where(input%3a%20IEnumerable).html>) 1.17 μs 698.77 ns 0.60 0.15 False
[WhereSingleOrDefault_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input%3a%20List).html>) 407.67 ns 304.07 ns 0.75 0.28 False
[Distinct - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Distinct(input%3a%20IEnumerable).html>) 2.97 μs 2.66 μs 0.90 0.09 False
[WhereAny_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input%3a%20IEnumerable).html>) 648.65 ns 364.51 ns 0.56 0.39 False
[GroupBy - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.GroupBy(input%3a%20IEnumerable).html>) 3.15 μs 2.31 μs 0.73 0.11 False
[FirstWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input%3a%20List).html>) 904.81 ns 366.18 ns 0.40 0.34 False
[SelectToList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SelectToList(input%3a%20IList).html>) 434.32 ns 181.54 ns 0.42 0.34 False
[TakeLastHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input%3a%20IEnumerable).html>) 1.72 μs 1.25 μs 0.73 0.10 True
[Where - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Where(input%3a%20List).html>) 971.08 ns 693.90 ns 0.71 0.12 False
[Where - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Where(input%3a%20Array).html>) 776.69 ns 611.07 ns 0.79 0.12 False
[Contains_ElementNotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input%3a%20IEnumerable).html>) 689.62 ns 196.73 ns 0.29 0.12 False
[WhereLast_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input%3a%20IEnumerable).html>) 653.26 ns 363.79 ns 0.56 0.31 False
[SelectToList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SelectToList(input%3a%20IEnumerable).html>) 1.17 μs 605.07 ns 0.52 0.00 True
[ToArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.ToArray(input%3a%20IEnumerable).html>) 1.14 μs 646.03 ns 0.57 0.05 True
[ToList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.ToList(input%3a%20ICollection).html>) 65.81 ns 59.57 ns 0.91 0.06 False
[SelectToList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SelectToList(input%3a%20Array).html>) 281.23 ns 169.09 ns 0.60 0.29 False
[Aggregate - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Aggregate(input%3a%20IEnumerable).html>) 855.36 ns 189.06 ns 0.22 0.08 True
[SequenceEqual - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1%3a%20IEnumerable%2c%20input2%3a%20IEnumerable).html>) 1.59 μs 497.06 ns 0.31 0.17 False
[Repeat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Repeat.html>) 698.01 ns 471.99 ns 0.68 0.01 True
[WhereSelect - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereSelect(input%3a%20Array).html>) 936.56 ns 632.48 ns 0.68 0.18 False
[WhereSingle_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input%3a%20IEnumerable).html>) 646.42 ns 358.90 ns 0.56 0.24 False
[Select - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Select(input%3a%20IEnumerable).html>) 1.15 μs 770.55 ns 0.67 0.14 False
[SingleWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input%3a%20List).html>) 911.92 ns 339.52 ns 0.37 0.35 False
[Intersect - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Intersect(input%3a%20IEnumerable).html>) 4.89 μs 3.63 μs 0.74 0.08 True
[Range - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Range.html>) 694.50 ns 446.91 ns 0.64 0.07 True
[WhereFirst_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input%3a%20List).html>) 402.47 ns 317.89 ns 0.79 0.27 False
[WhereLast_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input%3a%20List).html>) 408.05 ns 332.60 ns 0.82 0.27 False
[All_AllElementsMatch - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input%3a%20IEnumerable).html>) 846.79 ns 245.09 ns 0.29 0.08 True
[OrderBy - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.OrderBy(input%3a%20IEnumerable).html>) 3.54 μs 2.83 μs 0.80 0.31 False
[Aggregate_Seed - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Aggregate_Seed(input%3a%20IEnumerable).html>) 846.61 ns 190.88 ns 0.23 0.16 True
[WhereSingleOrDefault_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input%3a%20IEnumerable).html>) 681.83 ns 357.49 ns 0.52 0.31 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Count(input%3a%20ICollection).html>) 4.24 ns 3.01 ns 0.71 0.36 False
[AnyWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input%3a%20Array).html>) 859.30 ns 256.12 ns 0.30 0.06 True
[SelectToArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20IEnumerable).html>) 1.29 μs 621.54 ns 0.48 0.04 True
[SelectToList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SelectToList(input%3a%20List).html>) 283.56 ns 169.51 ns 0.60 0.28 False
[OrderByDescending - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.OrderByDescending(input%3a%20IEnumerable).html>) 4.38 μs 3.64 μs 0.83 0.25 False
[Max - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Max(input%3a%20IEnumerable).html>) 685.32 ns 196.94 ns 0.29 0.03 True
[SingleWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input%3a%20IEnumerable).html>) 869.00 ns 251.20 ns 0.29 0.07 True
[WhereAny_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input%3a%20List).html>) 406.40 ns 321.21 ns 0.79 0.29 False
[AnyWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input%3a%20List).html>) 918.60 ns 348.05 ns 0.38 0.35 False
[Sum - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Sum(input%3a%20IEnumerable).html>) 681.68 ns 192.74 ns 0.28 0.13 False
[WhereSingle_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input%3a%20List).html>) 424.05 ns 302.19 ns 0.71 0.30 False
[Except - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Except(input%3a%20IEnumerable).html>) 4.23 μs 3.26 μs 0.77 0.01 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Count(input%3a%20IEnumerable).html>) 482.59 ns 198.32 ns 0.41 0.07 True
[Append - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Append(input%3a%20IEnumerable).html>) 4.89 μs 4.07 μs 0.83 0.19 False
[WhereLast_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input%3a%20Array).html>) 250.55 ns 196.42 ns 0.78 0.24 False
[Select - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Select(input%3a%20List).html>) 956.76 ns 682.86 ns 0.71 0.12 True
[SingleWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input%3a%20Array).html>) 866.66 ns 241.31 ns 0.28 0.08 True
[LastWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input%3a%20IList).html>) 11.84 ns 8.37 ns 0.71 0.03 True
[Skip_One - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Skip_One(input%3a%20IEnumerable).html>) 1.09 μs 692.01 ns 0.64 0.13 False
[ElementAt - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.ElementAt(input%3a%20IEnumerable).html>) 244.16 ns 111.01 ns 0.45 0.05 True
[AnyWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input%3a%20IOrderedEnumerable).html>) 3.20 μs 1.97 μs 0.61 0.25 False
[Min - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Min(input%3a%20IEnumerable).html>) 670.88 ns 193.46 ns 0.29 0.14 False
[ToDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.ToDictionary(input%3a%20List).html>) 1.38 μs 1.06 μs 0.77 0.19 False
[FirstWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input%3a%20IOrderedEnumerable).html>) 2.88 μs 1.96 μs 0.68 0.23 False
[Select - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Select(input%3a%20IList).html>) 1.16 μs 771.84 ns 0.67 0.08 True
[SelectToArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20Range).html>) 268.48 ns 150.07 ns 0.56 0.24 False
[Contains_ElementNotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input%3a%20ICollection).html>) 23.35 ns 21.96 ns 0.94 0.02 False
[WhereSelect - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereSelect(input%3a%20IEnumerable).html>) 1.38 μs 802.15 ns 0.58 0.13 True
[Zip - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Zip(input%3a%20IEnumerable).html>) 2.00 μs 1.01 μs 0.50 0.08 True
[FirstWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input%3a%20Array).html>) 855.21 ns 257.52 ns 0.30 0.10 True
[EmptyTakeSelectToArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray.html>) 21.91 ns 17.92 ns 0.82 0.08 False
[SingleWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input%3a%20Array).html>) 864.41 ns 267.10 ns 0.31 0.10 True
[LastWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input%3a%20IEnumerable).html>) 861.92 ns 283.81 ns 0.33 0.05 True
[Take_All - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Take_All(input%3a%20IEnumerable).html>) 1.11 μs 721.48 ns 0.65 0.09 True
[SelectToArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20IList).html>) 423.54 ns 176.50 ns 0.42 0.34 False
[SingleWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input%3a%20List).html>) 1.47 μs 343.53 ns 0.23 0.44 False
[Prepend - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Prepend(input%3a%20IEnumerable).html>) 4.20 μs 3.46 μs 0.82 0.14 False
[WhereSingleOrDefault_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input%3a%20Array).html>) 248.25 ns 191.23 ns 0.77 0.36 False
[Concat_Once - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Concat_Once(input%3a%20IEnumerable).html>) 2.03 μs 1.61 μs 0.79 0.11 False
[ToList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.ToList(input%3a%20IEnumerable).html>) 943.86 ns 547.30 ns 0.58 0.13 True
[WhereFirst_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input%3a%20IEnumerable).html>) 650.53 ns 363.77 ns 0.56 0.32 False
[SequenceEqual - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1%3a%20Array%2c%20input2%3a%20Array).html>) 33.48 ns 24.36 ns 0.73 0.28 False
[Select - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Select(input%3a%20Array).html>) 798.38 ns 570.31 ns 0.71 0.08 False
[OrderByThenBy - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.OrderByThenBy(input%3a%20IEnumerable).html>) 5.03 μs 3.73 μs 0.74 0.32 False
[Concat_TenTimes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input%3a%20IEnumerable).html>) 11.34 μs 9.01 μs 0.79 0.08 False
[ToArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.ToArray(input%3a%20ICollection).html>) 54.88 ns 51.29 ns 0.93 0.02 False
[WhereSelect - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereSelect(input%3a%20List).html>) 1.11 μs 720.05 ns 0.65 0.20 False
[Reverse - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Reverse(input%3a%20IEnumerable).html>) 1.52 μs 1.07 μs 0.70 0.16 False
[ToDictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.ToDictionary(input%3a%20Array).html>) 1.10 μs 994.18 ns 0.90 0.07 False
[Average - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.Average(input%3a%20IEnumerable).html>) 690.60 ns 201.23 ns 0.29 0.10 True
[LastWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input%3a%20IOrderedEnumerable).html>) 1.31 μs 576.34 ns 0.44 0.12 True
[FirstWithPredicate_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input%3a%20IEnumerable).html>) 874.83 ns 258.77 ns 0.30 0.07 True
[WhereSingle_LastElementMatches - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input%3a%20Array).html>) 246.13 ns 190.77 ns 0.78 0.38 False
[TakeLastHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input%3a%20List).html>) 856.59 ns 531.10 ns 0.62 0.15 False
[SelectToList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.SelectToList(input%3a%20Range).html>) 270.74 ns 157.86 ns 0.58 0.25 False
[CastToSameType - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_Enumerable.CastToSameType(input%3a%20IEnumerable).html>) 719.45 ns 579.34 ns 0.81 0.08 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.SkipHalfTakeHalf(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 486.7592918622849 < 704.8573083726534. IsChangePoint: Marked as a change because one of 2/7/2023 3:56:56 PM, 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 77.87611971106713 (T) = (0 -487.0016228237256) / Math.Sqrt((3214.302169007242 / (299)) + (5.547407886435316 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34836731557831097 = (747.3560403986331 - 487.0016228237256) / 747.3560403986331 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 158.00191947954053 < 252.9966115962306. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.45850957471848 (T) = (0 -157.69675241130278) / Math.Sqrt((1600.4706610186104 / (299)) + (5.5486213957706 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.46256917277921866 = (293.4270689807668 - 157.69675241130278) / 293.4270689807668 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.2599183237902305 < 2.662492736505453. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 94.45972589680765 (T) = (0 -2259.1282846309455) / Math.Sqrt((11350.826178184014 / (299)) + (48.08539127554141 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21252500720401774 = (2868.825429757154 - 2259.1282846309455) / 2868.825429757154 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 256.72943939970247 < 815.7821660758967. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 484.44842231140706 (T) = (0 -257.83307040439) / Math.Sqrt((407.8153650594486 / (299)) + (2.28937486923453 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6998537958759324 = (859.0249247257276 - 257.83307040439) / 859.0249247257276 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 162.51921892020212 < 258.503451996133. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 61.73537950175584 (T) = (0 -159.7734700902243) / Math.Sqrt((2240.0173243400986 / (299)) + (1.5820652479858304 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5160082055552272 = (330.1160720576135 - 159.7734700902243) / 330.1160720576135 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 242.69804575597868 < 825.5878649395383. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 384.46687371475366 (T) = (0 -241.92166318446135) / Math.Sqrt((763.0419616259302 / (299)) + (0.651266032665884 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.719382098900997 = (862.1034589632633 - 241.92166318446135) / 862.1034589632633 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 698.7690127882195 < 1.1129628536269165. IsChangePoint: Marked as a change because one of 2/6/2023 10:25:44 PM, 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 83.40296038629239 (T) = (0 -698.1790521711359) / Math.Sqrt((10534.658004577645 / (299)) + (4.111161951112964 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4159715629606711 = (1195.453864730426 - 698.1790521711359) / 1195.453864730426 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 304.07433755476353 < 420.2189398889626. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 61.07090803165411 (T) = (0 -304.3544076435924) / Math.Sqrt((775.1126240954364 / (299)) + (11.785884382599827 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27290669645878624 = (418.59058000021963 - 304.3544076435924) / 418.59058000021963 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Distinct(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.6551732081911266 < 2.828195401746266. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.59949456782165 (T) = (0 -2647.668616804637) / Math.Sqrt((15814.400978149684 / (299)) + (171.13009310110507 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13444093874494487 = (3058.911558230971 - 2647.668616804637) / 3058.911558230971 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 364.51246295342537 < 618.2528395364249. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 101.70969110382357 (T) = (0 -365.1700322142784) / Math.Sqrt((2763.8427795281395 / (299)) + (3.213271416451511 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4618047448472186 = (678.5084571409218 - 365.1700322142784) / 678.5084571409218 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.GroupBy(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.3107593477531854 < 2.995953049399063. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.44068796108644 (T) = (0 -2338.8049517955574) / Math.Sqrt((11980.399092893067 / (299)) + (5527.620092735738 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25154282618914087 = (3124.834704819853 - 2338.8049517955574) / 3124.834704819853 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 366.18198102825426 < 855.1953471970158. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 53.57150826464727 (T) = (0 -364.09997437428905) / Math.Sqrt((42056.45244061146 / (299)) + (51.60010439326221 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6389166903356752 = (1008.3544839354904 - 364.09997437428905) / 1008.3544839354904 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 181.5352377989769 < 413.05297249611317. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 95.95281290999513 (T) = (0 -182.43793861205464) / Math.Sqrt((3903.5687386876975 / (299)) + (2.259678817770865 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6567086751912683 = (531.4376607498072 - 182.43793861205464) / 531.4376607498072 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2493244413381905 < 1.6357009882730402. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 122.26955005687755 (T) = (0 -1241.429603502157) / Math.Sqrt((4275.586352354138 / (299)) + (32.06331300800577 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2873860285055721 = (1742.0786753573552 - 1241.429603502157) / 1742.0786753573552 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Where(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 693.9018427354974 < 894.3876074665853. IsChangePoint: Marked as a change because one of 1/29/2023 4:03:01 PM, 1/30/2023 9:42:46 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 117.23958411200866 (T) = (0 -692.418969435826) / Math.Sqrt((1411.103260789386 / (299)) + (6.278249873892972 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2786000792667069 = (959.8267889078663 - 692.418969435826) / 959.8267889078663 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Where(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 611.0688530894556 < 743.0445614978574. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 89.09723624199704 (T) = (0 -611.3286746082584) / Math.Sqrt((1322.7293017214454 / (299)) + (0.7930414526182176 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2358524976563667 = (800.0139668497496 - 611.3286746082584) / 800.0139668497496 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 196.726355406622 < 656.4728699674037. IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 3/17/2023 1:20:12 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.13998883828287 (T) = (0 -197.44273761510024) / Math.Sqrt((14311.125998209653 / (299)) + (0.3411508074185627 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7247910621789821 = (717.4285078761036 - 197.44273761510024) / 717.4285078761036 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 363.7888600015757 < 630.9345353317407. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 105.43565926920289 (T) = (0 -365.21927889533407) / Math.Sqrt((2295.867090896762 / (299)) + (14.012765275161229 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4607049121468906 = (677.2160309288979 - 365.21927889533407) / 677.2160309288979 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SelectToList(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 605.0710955482318 < 1.1235814575235887. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 396.8167413817736 (T) = (0 -607.4412545821584) / Math.Sqrt((88.47041683978729 / (299)) + (24.38499144557702 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.49049218596547367 = (1192.2118520070346 - 607.4412545821584) / 1192.2118520070346 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 646.0334271028688 < 1.0807784571065382. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 204.81851781533803 (T) = (0 -638.500531273363) / Math.Sqrt((432.9519696212728 / (299)) + (57.23359203350815 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4369062565189576 = (1133.9151582934596 - 638.500531273363) / 1133.9151582934596 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.ToList(input: ICollection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 59.57306908264978 < 61.02427470778012. IsChangePoint: Marked as a change because one of 2/26/2023 10:10:21 PM, 4/25/2023 6:46:56 PM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.244539207935198 (T) = (0 -59.83912790835536) / Math.Sqrt((3.063118562176947 / (299)) + (0.5532476586046139 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.08773832653599754 = (65.59425836792714 - 59.83912790835536) / 65.59425836792714 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 169.09188164304695 < 328.9188419266687. IsChangePoint: Marked as a change because one of 4/26/2023 2:24:59 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 66.28686453735696 (T) = (0 -169.47162724467034) / Math.Sqrt((2720.6300044061154 / (299)) + (18.678416467490305 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5593933556474429 = (384.6324820945401 - 169.47162724467034) / 384.6324820945401 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Aggregate(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 189.0590906398843 < 816.9763506777489. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 557.1166575332145 (T) = (0 -189.85878761963528) / Math.Sqrt((411.06598590990444 / (299)) + (0.9898644554884412 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7794736988594246 = (860.9348936506624 - 189.85878761963528) / 860.9348936506624 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: IEnumerable, input2: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 497.057859738644 < 1.5081751753156423. IsChangePoint: Marked as a change because one of 2/6/2023 10:25:44 PM, 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 83.68275961222322 (T) = (0 -499.36819633211417) / Math.Sqrt((59497.152717072226 / (299)) + (14.182392242051277 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7032960624473397 = (1683.0521375992328 - 499.36819633211417) / 1683.0521375992328 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Repeat ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 471.9917953150077 < 665.1162925671663. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 458.4160912458011 (T) = (0 -471.7158486177869) / Math.Sqrt((61.4765061241677 / (299)) + (0.5296107243866236 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3253905091303105 = (699.2428286321065 - 471.7158486177869) / 699.2428286321065 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 632.4803901646007 < 907.8892198856563. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 126.74838982522012 (T) = (0 -633.5386242697822) / Math.Sqrt((2055.5659478372904 / (299)) + (4.320629157518357 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3494222726508481 = (973.8092738760096 - 633.5386242697822) / 973.8092738760096 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 358.90076978709044 < 616.3048948224979. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 124.70415437507013 (T) = (0 -358.6639938407177) / Math.Sqrt((1758.0547186671888 / (299)) + (3.3497070731559595 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4627598306473191 = (667.6045729657015 - 358.6639938407177) / 667.6045729657015 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 770.550946304631 < 1.0945997772166183. IsChangePoint: Marked as a change because one of 2/20/2023 2:00:07 PM, 2/23/2023 8:54:44 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 77.43548298356336 (T) = (0 -770.3689164061336) / Math.Sqrt((8572.685730299318 / (299)) + (2.2178221402871485 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35057522893022547 = (1186.2327258277076 - 770.3689164061336) / 1186.2327258277076 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 339.51748820672475 < 862.9593246378819. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.40542371310095 (T) = (0 -341.114217139413) / Math.Sqrt((45801.485274020124 / (299)) + (2.430870094257899 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6679403414771051 = (1027.2678670356875 - 341.114217139413) / 1027.2678670356875 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Intersect(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.634615551820079 < 4.66012923915434. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 163.45580111199922 (T) = (0 -3626.4539381483432) / Math.Sqrt((19902.223825670913 / (299)) + (103.52337727752536 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2801144037842595 = (5037.542016692249 - 3626.4539381483432) / 5037.542016692249 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Range ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 446.91130018586466 < 653.8751730984931. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 309.9158544755745 (T) = (0 -447.9441948931071) / Math.Sqrt((171.04973790346244 / (299)) + (0.46706602532762515 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35042824825747537 = (689.5992531871398 - 447.9441948931071) / 689.5992531871398 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 317.8877298003901 < 382.1360274092525. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.05766340727065 (T) = (0 -318.5721460488204) / Math.Sqrt((686.1156629838442 / (299)) + (20.698206269909978 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23289243562667863 = (415.29005949651116 - 318.5721460488204) / 415.29005949651116 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 332.60316816329475 < 392.98524013369484. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 45.81326619649015 (T) = (0 -310.85771267608663) / Math.Sqrt((747.9760198427906 / (299)) + (43.87139257761505 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26322047739623206 = (421.9141590384055 - 310.85771267608663) / 421.9141590384055 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 245.09173755233607 < 816.290863707783. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 555.1851947355433 (T) = (0 -246.88095924456331) / Math.Sqrt((233.26916505047507 / (299)) + (5.853889855102332 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7138365765580432 = (862.7271657400994 - 246.88095924456331) / 862.7271657400994 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.OrderBy(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.8309813649093694 < 3.372501027773197. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 72.76969201420283 (T) = (0 -2821.5166468250245) / Math.Sqrt((36232.0426570749 / (299)) + (77.20958738292322 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22527645675225036 = (3641.965797240176 - 2821.5166468250245) / 3641.965797240176 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Aggregate_Seed(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 190.87888473625102 < 812.4510619407229. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 421.989956591599 (T) = (0 -190.93668322056453) / Math.Sqrt((737.895204768646 / (299)) + (0.5057821922336952 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7777392071400353 = (859.0659682423797 - 190.93668322056453) / 859.0659682423797 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 357.4937693151232 < 623.2014208677656. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 116.90737047449845 (T) = (0 -357.02329669295955) / Math.Sqrt((2098.1286797892135 / (299)) + (2.0322019200038337 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4672410066299603 = (670.1403470161245 - 357.02329669295955) / 670.1403470161245 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Count(input: ICollection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.0079369611781246 < 4.171978727125538. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.20792885290893 (T) = (0 -2.944793612433336) / Math.Sqrt((0.0895071547597006 / (299)) + (0.011758531625966288 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3214564288179443 = (4.339874014727401 - 2.944793612433336) / 4.339874014727401 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 256.1153215305956 < 818.7386996828228. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 504.0589092922479 (T) = (0 -257.69942924178514) / Math.Sqrt((246.58950023480423 / (299)) + (7.675943173945492 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6994172722567421 = (857.3327921286898 - 257.69942924178514) / 857.3327921286898 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 621.5411821124565 < 1.219434993458184. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 253.90749414785554 (T) = (0 -621.0149429368312) / Math.Sqrt((815.1801509115982 / (299)) + (49.168724063294285 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5105420193320596 = (1268.7809116716435 - 621.0149429368312) / 1268.7809116716435 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SelectToList(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 169.51074199378453 < 270.40400441480386. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 57.95992886423908 (T) = (0 -168.14154095359436) / Math.Sqrt((5641.453460687196 / (299)) + (1.3531242484516621 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6002299248672305 = (420.59561586182275 - 168.14154095359436) / 420.59561586182275 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.OrderByDescending(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.644906199569661 < 4.153140393461913. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.02948613100817 (T) = (0 -3628.625914192962) / Math.Sqrt((51725.89828465851 / (299)) + (98.72233388672738 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19167700376483418 = (4489.079156591611 - 3628.625914192962) / 4489.079156591611 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 196.94459279255537 < 650.2916717742652. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 688.7020539335931 (T) = (0 -197.644382765555) / Math.Sqrt((141.1797742137307 / (299)) + (0.42703232559390836 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7123372202611175 = (687.0697103913162 - 197.644382765555) / 687.0697103913162 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 251.2022768843311 < 825.1523060390055. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 456.39074881467417 (T) = (0 -251.42226626293) / Math.Sqrt((489.2476731364263 / (299)) + (1.967763346983358 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7082015477186843 = (861.6298828773083 - 251.42226626293) / 861.6298828773083 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 321.2065366516598 < 386.46919330507995. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.616031687516944 (T) = (0 -320.91058430709376) / Math.Sqrt((862.4766328679699 / (299)) + (6.350619104491202 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.241443301830335 = (423.054183136771 - 320.91058430709376) / 423.054183136771 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 348.04977873939396 < 870.2530373917598. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.32872349700595 (T) = (0 -357.48041642566955) / Math.Sqrt((57221.214291663375 / (299)) + (105.09968079740868 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6515463630587754 = (1025.9052526002695 - 357.48041642566955) / 1025.9052526002695 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 192.74133719979602 < 650.6637225587073. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 77.65856761970169 (T) = (0 -192.39490615858023) / Math.Sqrt((13152.271028782077 / (299)) + (1.4979750728991543 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7283036479843525 = (708.1247309036374 - 192.39490615858023) / 708.1247309036374 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 302.1872326987022 < 393.3260171828705. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 67.96188915966613 (T) = (0 -302.8117631914273) / Math.Sqrt((863.4793400316703 / (299)) + (0.8544515167430097 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27835242358117435 = (419.61169563421794 - 302.8117631914273) / 419.61169563421794 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Except(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.2556057712327227 < 4.022665038354091. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 146.3878925301153 (T) = (0 -3277.449623431877) / Math.Sqrt((2778.592730991008 / (299)) + (473.9143921310652 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23201147794409574 = (4267.576310461189 - 3277.449623431877) / 4267.576310461189 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 198.3161672690257 < 459.05664579867556. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 312.067558998904 (T) = (0 -200.3682425450474) / Math.Sqrt((205.27824336324204 / (299)) + (1.9381691389141122 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5874136331662123 = (485.63951369184883 - 200.3682425450474) / 485.63951369184883 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Append(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.071797881539126 < 4.77147383617823. IsChangePoint: Marked as a change because one of 1/24/2023 11:24:10 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.653717631824314 (T) = (0 -3968.993691537418) / Math.Sqrt((248080.2713993455 / (299)) + (6167.071914954147 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23971685748392538 = (5220.4152237316 - 3968.993691537418) / 5220.4152237316 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 196.42372095736957 < 237.44520667644213. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.474531304527545 (T) = (0 -196.9683625809078) / Math.Sqrt((1020.1324481892412 / (299)) + (6.491912048648647 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2571372736704314 = (265.14772595216124 - 196.9683625809078) / 265.14772595216124 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Select(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 682.8573557523439 < 911.8121711349364. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 159.17720082641324 (T) = (0 -681.7702819396382) / Math.Sqrt((978.9653297231691 / (299)) + (1.6066720961583236 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30087762882711555 = (975.1801831142443 - 681.7702819396382) / 975.1801831142443 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 241.3080732814132 < 818.0145854493486. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 554.6500306480954 (T) = (0 -242.51158560873586) / Math.Sqrt((319.58008272384507 / (299)) + (2.616300509150148 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.720477060579582 = (867.5909966873421 - 242.51158560873586) / 867.5909966873421 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IList) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.36515092843178 < 11.246172775671194. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 114.56906271923381 (T) = (0 -8.31371920202125) / Math.Sqrt((0.133187397615302 / (299)) + (0.006375899975521464 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29656021166905705 = (11.818664994408799 - 8.31371920202125) / 11.818664994408799 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Skip_One(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 692.0134777922256 < 1.0202661435404135. IsChangePoint: Marked as a change because one of 1/30/2023 1:22:46 AM, 1/30/2023 9:42:46 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 69.64086103020745 (T) = (0 -700.3081320042895) / Math.Sqrt((8080.923894167406 / (299)) + (29.347262799680426 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3498613781210449 = (1077.1674046687772 - 700.3081320042895) / 1077.1674046687772 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.ElementAt(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 111.00787088168218 < 239.99429439487258. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 338.9115192918767 (T) = (0 -109.78268539066457) / Math.Sqrt((27.46179971658609 / (299)) + (1.2029516014871326 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5700018329851251 = (255.30965899877162 - 109.78268539066457) / 255.30965899877162 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IOrderedEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.9653513716647872 < 2.6977541243872176. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 51.74631112437524 (T) = (0 -1973.646349684235) / Math.Sqrt((30299.200829716556 / (299)) + (3272.2022156508633 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33004312957647564 = (2945.9304573390846 - 1973.646349684235) / 2945.9304573390846 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 193.45899387727832 < 641.3044419767724. IsChangePoint: Marked as a change because one of 3/15/2023 4:14:49 PM, 3/16/2023 10:20:48 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.38056712651334 (T) = (0 -193.15480470344912) / Math.Sqrt((35303.76017140488 / (299)) + (1.1318556792945296 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7392613052859541 = (740.7983878851718 - 193.15480470344912) / 740.7983878851718 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0583262948525054 < 1.1431914590325238. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.330066554442844 (T) = (0 -1069.4745129632472) / Math.Sqrt((1859.7543318947698 / (299)) + (83.01450628918236 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13579019405705128 = (1237.5172158528471 - 1069.4745129632472) / 1237.5172158528471 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.9592492798731804 < 2.692662699824092. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 91.44502806849611 (T) = (0 -1940.5946091514274) / Math.Sqrt((26336.18428854205 / (299)) + (420.26199664588836 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3408371821255364 = (2944.0292391022126 - 1940.5946091514274) / 2944.0292391022126 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Select(input: IList) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 771.8436589407127 < 1.0981538628105985. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 94.33806041985811 (T) = (0 -772.4257119913326) / Math.Sqrt((5463.308907974579 / (299)) + (1.2288569456646095 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3435784014752778 = (1176.7219630300472 - 772.4257119913326) / 1176.7219630300472 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Range) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 150.06512996275458 < 252.1963271867772. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.48704263725313 (T) = (0 -143.90401892137635) / Math.Sqrt((1426.5078095590245 / (299)) + (19.34708864322873 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4910075990764114 = (282.7233150441074 - 143.90401892137635) / 282.7233150441074 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 21.957082723325865 < 22.27289835664412. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 8.716541426633603 (T) = (0 -21.625099297630012) / Math.Sqrt((0.0932858538321213 / (299)) + (0.33095227604968486 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06077326456324881 = (23.024365131145988 - 21.625099297630012) / 23.024365131145988 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 802.1529702754963 < 1.3039876184303796. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 125.48722028836426 (T) = (0 -780.465394765657) / Math.Sqrt((5785.008510848315 / (299)) + (45.31055758010992 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4344846377063415 = (1380.0958325874446 - 780.465394765657) / 1380.0958325874446 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Zip(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0053413758252154 < 1.905578472633649. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 207.16454854770927 (T) = (0 -1005.1890859572972) / Math.Sqrt((6228.565832039317 / (299)) + (3.0721687888681046 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4861164858608121 = (1956.064085147975 - 1005.1890859572972) / 1956.064085147975 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 257.52254875178346 < 817.7612047934699. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 382.2869926451508 (T) = (0 -259.33365173106876) / Math.Sqrt((565.3272476959329 / (299)) + (7.096496355399527 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6970649584406438 = (856.0701673736731 - 259.33365173106876) / 856.0701673736731 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.92285854864485 < 20.808950947580286. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.241781617504174 (T) = (0 -18.051073850418224) / Math.Sqrt((0.19688481081907677 / (299)) + (0.08190593043334017 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18229964903337934 = (22.075414091579724 - 18.051073850418224) / 22.075414091579724 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 267.1035576239559 < 820.2851130945248. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 292.39162838546986 (T) = (0 -253.43859510245395) / Math.Sqrt((825.6635656281466 / (299)) + (18.813380220018033 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7029817865030097 = (853.2762759514143 - 253.43859510245395) / 853.2762759514143 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 283.8058152712849 < 828.0322372426286. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 512.6161289380136 (T) = (0 -284.1647608629812) / Math.Sqrt((377.38577528303824 / (299)) + (0.6343555832089697 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6737843691297706 = (871.0948647829315 - 284.1647608629812) / 871.0948647829315 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Take_All(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 721.4826826241643 < 1.0536830013620846. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 90.11614611994678 (T) = (0 -722.9836596454752) / Math.Sqrt((5694.238080889409 / (299)) + (2.403462917579654 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35341231139924456 = (1118.1525296438044 - 722.9836596454752) / 1118.1525296438044 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 176.49682785697684 < 403.1640288355751. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 117.79929178918266 (T) = (0 -176.37843159242297) / Math.Sqrt((1578.6943209328476 / (299)) + (3.0943767729168976 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6107231536327716 = (453.09253103133335 - 176.37843159242297) / 453.09253103133335 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 343.5296118597286 < 1.132389328527331. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 52.156480662465654 (T) = (0 -342.1640221572015) / Math.Sqrt((54455.30063803217 / (299)) + (0.5299898262682037 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6729187725625083 = (1046.1132998609414 - 342.1640221572015) / 1046.1132998609414 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Prepend(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.4623195429035594 < 3.991512044004704. IsChangePoint: Marked as a change because one of 2/12/2023 9:02:44 AM, 2/13/2023 12:37:33 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.46577277341874 (T) = (0 -3504.0148959795206) / Math.Sqrt((119004.41253401164 / (299)) + (1336.8511817795586 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20537020868779401 = (4409.6193400869 - 3504.0148959795206) / 4409.6193400869 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 191.2284607178715 < 234.8897594983321. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.69346938521238 (T) = (0 -190.52837683075293) / Math.Sqrt((1190.5189943788941 / (299)) + (0.38692468780722705 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27835658339468766 = (264.0201135998978 - 190.52837683075293) / 264.0201135998978 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.605711233920458 < 1.9347507064028775. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 42.482502707071724 (T) = (0 -1597.607480942554) / Math.Sqrt((33656.644050687035 / (299)) + (21.73430775702683 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22131193056220697 = (2051.665543169314 - 1597.607480942554) / 2051.665543169314 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.ToList(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 547.2977866542476 < 894.736085542423. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 195.21168217566623 (T) = (0 -537.0932818655775) / Math.Sqrt((1081.7210642251237 / (299)) + (19.93098723834171 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4520238604220921 = (980.1399058712425 - 537.0932818655775) / 980.1399058712425 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 363.76684612303575 < 618.5015890039494. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 108.51171035115286 (T) = (0 -360.7873044622544) / Math.Sqrt((2324.9997221405633 / (299)) + (3.36120087837585 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.46019446185318724 = (668.3653259669408 - 360.7873044622544) / 668.3653259669408 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: Array, input2: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 24.359866281604813 < 27.304967293030945. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.4674183007445 (T) = (0 -24.274594786994026) / Math.Sqrt((9.283575531214463 / (299)) + (0.04560744258643393 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18411429786170094 = (29.75244537730518 - 24.274594786994026) / 29.75244537730518 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Select(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 570.3110273483003 < 756.5106915236499. IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 81.3649109277249 (T) = (0 -570.5932144093524) / Math.Sqrt((6538.761391483867 / (299)) + (7.0418398966342375 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4030028236685391 = (955.7720488991918 - 570.5932144093524) / 955.7720488991918 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.OrderByThenBy(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.7265812919491665 < 4.852193730565221. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.03639691163203 (T) = (0 -3655.5844642412944) / Math.Sqrt((29428.675609614464 / (299)) + (11754.49315917748 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2938270978143082 = (5176.613904224889 - 3655.5844642412944) / 5176.613904224889 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.008050930510851 < 10.835290574021013. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 44.547277908770084 (T) = (0 -8924.543995432108) / Math.Sqrt((936921.2752449551 / (299)) + (1147.3744549886908 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2207733208329933 = (11453.078076038728 - 8924.543995432108) / 11453.078076038728 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.ToArray(input: ICollection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 51.2933527797691 < 52.71618112866655. IsChangePoint: Marked as a change because one of 3/1/2023 11:13:23 AM, 4/25/2023 1:02:54 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.2659086691417 (T) = (0 -51.18026333689459) / Math.Sqrt((2.1361610945379383 / (299)) + (0.35820004714660364 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1021684663737342 = (57.00430584141083 - 51.18026333689459) / 57.00430584141083 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 720.0522621909341 < 1.0608157508322853. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 135.32038103114058 (T) = (0 -719.9961778994713) / Math.Sqrt((2740.5171367220105 / (299)) + (4.394819329303473 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3668483059089648 = (1137.1622071281222 - 719.9961778994713) / 1137.1622071281222 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Reverse(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0692243413516613 < 1.4622959134815623. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 81.51273010698338 (T) = (0 -1071.445180616285) / Math.Sqrt((8848.741443083158 / (299)) + (33.76734680234303 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30150395713783784 = (1533.931640079053 - 1071.445180616285) / 1533.931640079053 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 994.1784688013137 < 1.047394024703625. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.33305875469632 (T) = (0 -1010.3854744252606) / Math.Sqrt((299.2565939911691 / (299)) + (106.00042769662211 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1101422581230347 = (1135.4460683727577 - 1010.3854744252606) / 1135.4460683727577 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.Average(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 201.22633654253627 < 654.8686367533131. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 439.4995866057802 (T) = (0 -202.1050989303934) / Math.Sqrt((320.7843542143356 / (299)) + (1.9364871240898165 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7062040722090883 = (687.9098034137058 - 202.1050989303934) / 687.9098034137058 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IOrderedEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 576.3369144821659 < 1.2193914859257036. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 319.10779589957446 (T) = (0 -581.1404135930804) / Math.Sqrt((1175.57906380098 / (299)) + (15.039885465668554 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5533065407086393 = (1300.9825899734635 - 581.1404135930804) / 1300.9825899734635 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 258.7738514984963 < 811.7012310293754. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 579.3576683070139 (T) = (0 -259.853247591257) / Math.Sqrt((173.15610430985518 / (299)) + (6.615062640801359 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6992983764814067 = (864.1564503398483 - 259.853247591257) / 864.1564503398483 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 190.7733107331426 < 233.91104025611838. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.56425842824698 (T) = (0 -191.0466089968362) / Math.Sqrt((1169.430575529785 / (299)) + (0.8785165125118158 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27628465071534664 = (263.9803193143183 - 191.0466089968362) / 263.9803193143183 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: List) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 531.09536934619 < 815.3067121993679. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.43684869545767 (T) = (0 -527.889578102109) / Math.Sqrt((5609.448240023972 / (299)) + (14.225022522157486 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.38903624534355385 = (864.0276515240238 - 527.889578102109) / 864.0276515240238 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.SelectToList(input: Range) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 157.86390392649358 < 260.6071049359551. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 67.6878243876158 (T) = (0 -150.94673950667467) / Math.Sqrt((3407.8049155407025 / (299)) + (7.829226937733464 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6083580230576873 = (385.4202266191413 - 150.94673950667467) / 385.4202266191413 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 579.3406435231126 < 683.6154293092047. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 98.14414156153325 (T) = (0 -579.5146617774418) / Math.Sqrt((585.770891079125 / (299)) + (4.128021759851813 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2035255706475944 = (727.5998329897815 - 579.5146617774418) / 727.5998329897815 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToStream(Mode%3a%20SourceGen).html>) 310.04 ns 249.85 ns 0.81 0.12 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToString(Mode%3a%20SourceGen).html>) 248.50 ns 196.53 ns 0.79 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 220.73 ns 169.75 ns 0.77 0.03 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToStream(Mode%3a%20Reflection).html>) 311.27 ns 241.45 ns 0.78 0.11 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 392.93 ns 313.01 ns 0.80 0.01 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeObjectProperty(Mode%3a%20Reflection).html>) 394.24 ns 316.92 ns 0.80 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 221.41 ns 166.75 ns 0.75 0.02 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToWriter(Mode%3a%20SourceGen).html>) 123.15 ns 97.76 ns 0.79 0.00 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToString(Mode%3a%20Reflection).html>) 247.60 ns 203.15 ns 0.82 0.00 True
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToWriter(Mode%3a%20Reflection).html>) 124.03 ns 100.53 ns 0.81 0.01 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 249.8461211765389 < 295.58177865448334. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 69.66181090260748 (T) = (0 -243.4741546572289) / Math.Sqrt((70.39431851200553 / (299)) + (11.246221449812202 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23085952760247466 = (316.55355997362057 - 243.4741546572289) / 316.55355997362057 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 196.52782952005379 < 237.383475426091. IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 4/17/2023 5:51:51 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.90507433460884 (T) = (0 -200.8373043216716) / Math.Sqrt((46.13400552031402 / (299)) + (13.039823945939808 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1797309714761863 = (244.8432128214152 - 200.8373043216716) / 244.8432128214152 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 169.75192217950712 < 200.94258727283867. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 6:03:21 AM, 4/8/2023 3:09:32 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 58.34098275841772 (T) = (0 -169.16357845193286) / Math.Sqrt((199.04790240107724 / (299)) + (1.5341740506905444 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23390168705402048 = (220.81184045612332 - 169.16357845193286) / 220.81184045612332 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 241.4528281986376 < 289.85037356007723. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.66678513992404 (T) = (0 -243.1036310789719) / Math.Sqrt((74.59378035159908 / (299)) + (8.251759105437278 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22641454372275466 = (314.25568966726536 - 243.1036310789719) / 314.25568966726536 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 313.00724050655083 < 376.90174822708656. IsChangePoint: Marked as a change because one of 2/6/2023 10:25:44 PM, 4/12/2023 2:59:13 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.8743871587278 (T) = (0 -316.7958489986291) / Math.Sqrt((85.4583371130591 / (299)) + (19.160750115688845 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1896056218455528 = (390.9156548199218 - 316.7958489986291) / 390.9156548199218 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 316.9165613131979 < 373.67064528080806. IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.56988508274429 (T) = (0 -319.21478204472226) / Math.Sqrt((94.3924421634925 / (299)) + (7.857337147037275 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18505909136672516 = (391.7029795204079 - 319.21478204472226) / 391.7029795204079 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 166.75023166076565 < 209.50634473233234. IsChangePoint: Marked as a change because one of 3/20/2023 7:45:43 PM, 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 91.08755050703726 (T) = (0 -167.96177079262276) / Math.Sqrt((43.19051273931119 / (299)) + (2.097636954564951 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23070822157653922 = (218.33298561546215 - 167.96177079262276) / 218.33298561546215 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 97.76343402506124 < 118.2839676875859. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 87.79191201299939 (T) = (0 -98.18760899510208) / Math.Sqrt((11.697943253505164 / (299)) + (0.7084191715911642 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.214808536971638 = (125.04925692442919 - 98.18760899510208) / 125.04925692442919 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 203.14618773650557 < 234.00182866199668. IsChangePoint: Marked as a change because one of 3/20/2023 7:45:43 PM, 4/8/2023 9:16:19 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 65.11312057146486 (T) = (0 -195.45623854204652) / Math.Sqrt((34.76126968467945 / (299)) + (5.843961711488593 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20037166055310804 = (244.43385620530265 - 195.45623854204652) / 244.43385620530265 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 100.53339557652285 < 119.12382162446673. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 68.99965876863001 (T) = (0 -99.61712349448422) / Math.Sqrt((10.0565744550063 / (299)) + (1.3730076389159664 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2053844683705631 = (125.36518546297422 - 99.61712349448422) / 125.36518546297422 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Email_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options%3a%20None).html>) 438.25 ns 402.96 ns 0.92 0.00 True
[ReplaceWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options%3a%20IgnoreCase%2c%20Compiled).html>) 4.47 μs 4.22 μs 0.94 0.01 False
[MatchesWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options%3a%20None).html>) 72.06 μs 61.49 μs 0.85 0.00 True
[IP_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options%3a%20None).html>) 788.48 ns 634.91 ns 0.81 0.00 True
[Backtracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options%3a%20None).html>) 2.06 μs 1.54 μs 0.75 0.01 True
[Date_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options%3a%20Compiled).html>) 78.00 ns 72.74 ns 0.93 0.04 False
[ReplaceWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options%3a%20None).html>) 71.78 μs 61.06 μs 0.85 0.01 True
[Email_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options%3a%20None).html>) 655.16 ns 534.85 ns 0.82 0.00 True
[MatchesWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options%3a%20None).html>) 1.92 μs 1.76 μs 0.92 0.06 False
[Uri_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options%3a%20None).html>) 366.46 ns 341.68 ns 0.93 0.00 True
[OneNodeBacktracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options%3a%20None).html>) 1.32 μs 1.11 μs 0.84 0.00 True
[MatchWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options%3a%20None).html>) 2.13 μs 1.72 μs 0.81 0.00 True
[Date_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options%3a%20None).html>) 473.27 ns 391.92 ns 0.83 0.00 True
[MatchesBoundary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options%3a%20None).html>) 151.21 μs 136.19 μs 0.90 0.00 True
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options%3a%20Compiled).html>) 30.97 μs 25.60 μs 0.83 0.36 False
[Uri_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options%3a%20None).html>) 325.70 ns 291.38 ns 0.89 0.00 True
[Date_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options%3a%20None).html>) 188.55 ns 167.34 ns 0.89 0.00 True
[ReplaceWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options%3a%20Compiled).html>) 3.95 μs 3.64 μs 0.92 0.02 False
[SplitWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options%3a%20Compiled).html>) 4.04 μs 3.75 μs 0.93 0.00 True
[IP_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options%3a%20None).html>) 785.12 ns 635.75 ns 0.81 0.00 True
[MatchesSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options%3a%20None).html>) 169.96 μs 146.37 μs 0.86 0.00 True
[SplitWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options%3a%20None).html>) 72.31 μs 61.42 μs 0.85 0.00 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 402.96442550528434 < 417.9360548543357. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 46.500847847699056 (T) = (0 -402.9890813062482) / Math.Sqrt((5.239652284553794 / (299)) + (8.25636586021655 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08526953867698589 = (440.5550031901066 - 402.9890813062482) / 440.5550031901066 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.219198537933494 < 4.282035934104458. IsChangePoint: Marked as a change because one of 4/30/2023 6:17:41 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.760440884285416 (T) = (0 -4209.542343715444) / Math.Sqrt((22658.301328512593 / (299)) + (1043.4561101320162 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1241357171992314 = (4806.158244350948 - 4209.542343715444) / 4806.158244350948 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 61.48810947910356 < 68.527746151087. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 139.85912800625636 (T) = (0 -61254.254528653546) / Math.Sqrt((139769.13191821796 / (299)) + (74506.15563388384 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1523734643393819 = (72265.61693342171 - 61254.254528653546) / 72265.61693342171 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 634.9131373424831 < 747.8781877395078. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 234.31322598510295 (T) = (0 -635.3172739985165) / Math.Sqrt((13.099314543342823 / (299)) + (4.8031779072362655 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19165781812224061 = (785.9509106931545 - 635.3172739985165) / 785.9509106931545 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.5363686194324002 < 1.9370243858152112. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 128.4404384383749 (T) = (0 -1532.2039668802597) / Math.Sqrt((815.099308893184 / (299)) + (155.65276217886878 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24322211046016623 = (2024.6415600380865 - 1532.2039668802597) / 2024.6415600380865 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 72.73636108559695 < 73.80004853366945. IsChangePoint: Marked as a change because one of 3/28/2023 2:42:21 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.290059882515589 (T) = (0 -73.2347909473572) / Math.Sqrt((4.453215662361418 / (299)) + (2.0500119200394615 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.050061132164318 = (77.09421461426628 - 73.2347909473572) / 77.09421461426628 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 61.05678362165178 < 68.1576233062893. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.11835568017914 (T) = (0 -61114.40397156042) / Math.Sqrt((1853442.8344723142 / (299)) + (429168.43156999204 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1538558130026526 = (72226.93828156266 - 61114.40397156042) / 72226.93828156266 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 534.8505149673097 < 623.0682762012972. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 232.39584661176303 (T) = (0 -534.8364375792559) / Math.Sqrt((18.69204742945381 / (299)) + (2.881311332148673 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1880642828882086 = (658.7177116456584 - 534.8364375792559) / 658.7177116456584 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7591144128381684 < 1.8223174327499598. IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 42.73479123966845 (T) = (0 -1759.9970634403242) / Math.Sqrt((9253.322306222197 / (299)) + (36.52092219506032 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12363357977836147 = (2008.289024806781 - 1759.9970634403242) / 2008.289024806781 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 341.6759623616965 < 349.2977146574628. IsChangePoint: Marked as a change because one of 2/24/2023 10:36:11 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.17936419808686 (T) = (0 -339.21457107636314) / Math.Sqrt((6.723105297748908 / (299)) + (2.7647225993603755 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08403816699569258 = (370.33701498648355 - 339.21457107636314) / 370.33701498648355 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.107545630399696 < 1.2557740951372411. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 82.84318959445878 (T) = (0 -1100.456644824906) / Math.Sqrt((40.87113824242223 / (299)) + (89.78269490392069 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16651781467130106 = (1320.3121364746637 - 1100.456644824906) / 1320.3121364746637 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7233892833514486 < 1.9634211057050088. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 188.18634146109352 (T) = (0 -1726.19942124033) / Math.Sqrt((455.8201451316117 / (299)) + (19.134190195393256 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.158751300804223 = (2051.948993074883 - 1726.19942124033) / 2051.948993074883 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 391.9193852029673 < 451.1255665485349. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 71.4972271817819 (T) = (0 -390.4738061771259) / Math.Sqrt((3.154612941788262 / (299)) + (18.034028838087313 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1779558928098892 = (475.0034733681542 - 390.4738061771259) / 475.0034733681542 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 136.19193322981366 < 143.6860427849608. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 118.74458573786058 (T) = (0 -136427.14436689872) / Math.Sqrt((983991.5828672935 / (299)) + (176886.62300444502 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10164238752437522 = (151862.84667966835 - 136427.14436689872) / 151862.84667966835 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 25.600426151761518 < 29.02516300475971. IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.991258633344582 (T) = (0 -26260.340204668973) / Math.Sqrt((1345972.7834270927 / (299)) + (895010.3906305204 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11005805165965014 = (29507.92493110568 - 26260.340204668973) / 29507.92493110568 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 291.3844552123166 < 309.8964380879056. IsChangePoint: Marked as a change because one of 2/17/2023 6:56:12 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 101.53778905559062 (T) = (0 -290.74757213350114) / Math.Sqrt((3.617604683868349 / (299)) + (1.6261576343604578 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11453568750358754 = (328.3560591095862 - 290.74757213350114) / 328.3560591095862 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 167.33666906835978 < 180.1798110549403. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 104.11219395531378 (T) = (0 -167.53932525802526) / Math.Sqrt((1.4640691839200646 / (299)) + (0.5341278018799409 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11758640791518775 = (189.86485108665732 - 167.53932525802526) / 189.86485108665732 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.6396036513545353 < 3.8043317315408123. IsChangePoint: Marked as a change because one of 4/30/2023 6:17:41 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.40060648890547 (T) = (0 -3655.9978901245654) / Math.Sqrt((14024.216143036192 / (299)) + (996.7348608270757 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13288420612587779 = (4216.274130805765 - 3655.9978901245654) / 4216.274130805765 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.7471783069415054 < 3.861164345655781. IsChangePoint: Marked as a change because one of 3/15/2023 2:24:48 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.31028611786097 (T) = (0 -3748.2559163920805) / Math.Sqrt((1255.5711881253217 / (299)) + (473.32258267181624 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0873700469604708 = (4107.092807888293 - 3748.2559163920805) / 4107.092807888293 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 635.7453458350319 < 746.8212678225208. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 314.6893531644559 (T) = (0 -635.5040280153527) / Math.Sqrt((13.401889373180833 / (299)) + (2.3816867600993734 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19123991317483546 = (785.7757057597405 - 635.5040280153527) / 785.7757057597405 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 146.36744791666663 < 160.89438916168893. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 63.828120455603866 (T) = (0 -147295.6147575889) / Math.Sqrt((1385144.9064469743 / (299)) + (1615370.9543848394 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13462866781443594 = (170210.87858963633 - 147295.6147575889) / 170210.87858963633 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 61.41944281045752 < 69.5409247570306. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 152.60861838323254 (T) = (0 -61297.748589247) / Math.Sqrt((306503.9812971758 / (299)) + (53884.76477049812 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15183250113229552 = (72270.80579140194 - 61297.748589247) / 72270.80579140194 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromString(Mode%3a%20Reflection).html>) 207.57 ns 183.44 ns 0.88 0.05 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 148.93 ns 136.51 ns 0.92 0.05 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromReader(Mode%3a%20Reflection).html>) 255.94 ns 229.61 ns 0.90 0.22 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromStream(Mode%3a%20SourceGen).html>) 431.56 ns 368.08 ns 0.85 0.08 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromString(Mode%3a%20SourceGen).html>) 210.28 ns 184.56 ns 0.88 0.03 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 152.21 ns 135.96 ns 0.89 0.06 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromStream(Mode%3a%20Reflection).html>) 431.00 ns 385.87 ns 0.90 0.09 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 183.44459583764518 < 197.28084239349232. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 78.26951703228256 (T) = (0 -184.82740802161908) / Math.Sqrt((12.205673832881516 / (299)) + (0.8840662428198311 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12257554669236194 = (210.64766012033613 - 184.82740802161908) / 210.64766012033613 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 136.51238594127275 < 142.17349277993483. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.90512770751387 (T) = (0 -138.7022669707462) / Math.Sqrt((12.291015031398214 / (299)) + (4.151496360814714 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07564160520571701 = (150.05247721216892 - 138.7022669707462) / 150.05247721216892 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 229.61463975868327 < 244.290105532672. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 7.011513038125297 (T) = (0 -231.72294181675923) / Math.Sqrt((151.74972535751846 / (299)) + (225.12660339999317 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11327773332851887 = (261.32527683846865 - 231.72294181675923) / 261.32527683846865 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 368.0826696928552 < 410.9090594402516. IsChangePoint: Marked as a change because one of 2/8/2023 4:45:44 PM, 4/19/2023 10:34:24 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.561933342162295 (T) = (0 -380.8979305937758) / Math.Sqrt((181.98651979929662 / (299)) + (193.7213573010105 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12297950530836908 = (434.3090416920112 - 380.8979305937758) / 434.3090416920112 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 184.562591015447 < 203.5425002740487. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.839034333656805 (T) = (0 -187.89692175243576) / Math.Sqrt((14.49374552947057 / (299)) + (17.170716423195532 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10996556829714564 = (211.11196944700535 - 187.89692175243576) / 211.11196944700535 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 135.96322633125402 < 144.3304755953181. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.597197908309056 (T) = (0 -137.81914689338674) / Math.Sqrt((15.978682047072015 / (299)) + (2.84546207574412 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08520073367974518 = (150.65506933314344 - 137.81914689338674) / 150.65506933314344 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 385.8749013959937 < 409.15140163924707. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 3/11/2023 8:53:40 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.18033653518725 (T) = (0 -368.48774876652686) / Math.Sqrt((159.41693139667734 / (299)) + (51.03154136777433 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15569345548105187 = (436.4383423990588 - 368.48774876652686) / 436.4383423990588 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EncodeUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments%3a%20Url%2c%c2%a92020%2c16).html>) 42.74 ns 38.67 ns 0.90 0.00 False
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20UnsafeRelaxed%2cno%20(escaping%20%2f)%20required%2c16).html>) 17.39 ns 15.84 ns 0.91 0.08 False
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20JavaScript%2cno%20escaping%20required%2c16).html>) 18.39 ns 17.06 ns 0.93 0.08 False
[EncodeUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments%3a%20UnsafeRelaxed%2cno%20(escaping%20%2f)%20required%2c16).html>) 22.14 ns 19.46 ns 0.88 0.04 False
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20Url%2c%c2%a92020%2c16).html>) 48.62 ns 42.12 ns 0.87 0.00 False
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20Url%2c%c2%a92020%2c512).html>) 101.69 ns 92.95 ns 0.91 0.01 False
[EncodeUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments%3a%20JavaScript%2cno%20escaping%20required%2c512).html>) 73.95 ns 68.85 ns 0.93 0.04 False
[EncodeUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments%3a%20JavaScript%2cno%20escaping%20required%2c16).html>) 22.16 ns 19.84 ns 0.90 0.02 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,©2020,16) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 38.66682384918532 < 40.58231520177611. IsChangePoint: Marked as a change because one of 3/16/2023 1:49:47 PM, 4/3/2023 6:15:03 PM, 5/1/2023 3:45:46 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.578743322968784 (T) = (0 -38.33711428914768) / Math.Sqrt((0.7128443361782771 / (299)) + (0.15837230503396602 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0904257986604819 = (42.148418713601494 - 38.33711428914768) / 42.148418713601494 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no <escaping /> required,16) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.841587718192557 < 16.52754047090523. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.2166155494739 (T) = (0 -15.945895702986357) / Math.Sqrt((0.20422713034933265 / (299)) + (0.07967661010792321 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10313232225123334 = (17.779541061187814 - 15.945895702986357) / 17.779541061187814 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,16) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.057543495356846 < 17.581118396770993. IsChangePoint: Marked as a change because one of 2/1/2023 6:25:50 PM, 2/7/2023 3:56:56 PM, 2/14/2023 11:21:41 AM, 3/10/2023 4:14:40 PM, 4/26/2023 12:55:36 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.93376922210514 (T) = (0 -16.93479757762737) / Math.Sqrt((0.1917020555830061 / (299)) + (0.11068795632503846 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08732401710002702 = (18.555103777156567 - 16.93479757762737) / 18.555103777156567 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,16) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.461651439407 < 20.9849862407172. IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.95188988179972 (T) = (0 -19.334466288595284) / Math.Sqrt((0.1721056184523055 / (299)) + (0.22184826378710598 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1097826834351593 = (21.718816213554323 - 19.334466288595284) / 21.718816213554323 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,©2020,16) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 42.11680363654008 < 46.09469762038616. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.44519353131199 (T) = (0 -43.69354350565253) / Math.Sqrt((0.4091569788351918 / (299)) + (0.2490394088250213 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08804695370443263 = (47.912053897006544 - 43.69354350565253) / 47.912053897006544 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,©2020,512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 92.95483152384544 < 97.93028072861048. IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 3/10/2023 4:14:40 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 88.78607641014182 (T) = (0 -93.00472708224581) / Math.Sqrt((4.739144907892076 / (299)) + (0.0045367246905662125 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10833863991541645 = (104.30498757221388 - 93.00472708224581) / 104.30498757221388 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 68.8548830642643 < 70.18933499289007. IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 3/10/2023 4:14:40 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 35.426895238757126 (T) = (0 -68.28150990385257) / Math.Sqrt((4.8127494563566176 / (299)) + (0.29705551833078586 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09288138841273645 = (75.27296764904264 - 68.28150990385257) / 75.27296764904264 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,16) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.843498110154858 < 21.008416213834728. IsChangePoint: Marked as a change because one of 4/22/2023 7:07:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.447676379439592 (T) = (0 -19.32882157717428) / Math.Sqrt((0.15615208139469797 / (299)) + (0.06964612620780325 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11091764059028568 = (21.74019242717537 - 19.32882157717428) / 21.74019242717537 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueTrue(Int32%2c%20Int32).SortedList(Size%3a%20512).html>) 28.78 μs 24.52 μs 0.85 0.03 False
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueTrue(Int32%2c%20Int32).ImmutableDictionary(Size%3a%20512).html>) 21.54 μs 13.17 μs 0.61 0.07 True
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueTrue(Int32%2c%20Int32).SortedDictionary(Size%3a%20512).html>) 40.21 μs 28.69 μs 0.71 0.12 False
[FrozenDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueTrue(Int32%2c%20Int32).FrozenDictionary(Size%3a%20512).html>) 7.23 μs 5.98 μs 0.83 0.03 True
[IDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueTrue(Int32%2c%20Int32).IDictionary(Size%3a%20512).html>) 6.65 μs 5.77 μs 0.87 0.02 False
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueTrue(Int32%2c%20Int32).ConcurrentDictionary(Size%3a%20512).html>) 5.89 μs 3.51 μs 0.60 0.14 False
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueTrue(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 30.42 μs 19.07 μs 0.63 0.19 False
[FrozenDictionaryOptimized - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueTrue(Int32%2c%20Int32).FrozenDictionaryOptimized(Size%3a%20512).html>) 3.53 μs 2.82 μs 0.80 0.01 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueTrue<Int32, Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 24.520505581516094 < 27.34088758042279. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.498009605673445 (T) = (0 -23958.174717108704) / Math.Sqrt((577461.8545174825 / (299)) + (917877.9083621862 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20185995602478715 = (30017.507451177018 - 23958.174717108704) / 30017.507451177018 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueTrue<Int32, Int32>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.168739706599977 < 20.423790553838746. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 113.65628534408246 (T) = (0 -13143.083736724224) / Math.Sqrt((72563.98163940292 / (299)) + (68131.8126475843 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3903803382812235 = (21559.48136526354 - 13143.083736724224) / 21559.48136526354 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueTrue<Int32, Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 28.687579510703365 < 38.21841801663022. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 82.49296509236959 (T) = (0 -28783.590546128613) / Math.Sqrt((7386400.837030097 / (299)) + (9612.113716256676 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31372787348692904 = (41941.94902301688 - 28783.590546128613) / 41941.94902301688 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueTrue<Int32, Int32>.FrozenDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.97879133107249 < 6.859153564602256. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 26.894298164522557 (T) = (0 -6049.023637476546) / Math.Sqrt((36216.41685620628 / (172)) + (27789.73892905518 / (13))) is greater than 1.9730119151360246 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (172) + (13) - 2, .975) and 0.1772591185467063 = (7352.282807184104 - 6049.023637476546) / 7352.282807184104 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueTrue<Int32, Int32>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.772479357798165 < 6.3253510507909425. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 35.0535756731948 (T) = (0 -5953.694038767539) / Math.Sqrt((41650.01544225786 / (299)) + (13497.76558029101 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16808254620326624 = (7156.592293617429 - 5953.694038767539) / 7156.592293617429 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueTrue<Int32, Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.5063290999647108 < 5.598724636758916. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 101.68343509352728 (T) = (0 -3509.935256642997) / Math.Sqrt((213590.6379157727 / (299)) + (79.78701132842629 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4374483077160845 = (6239.311524231554 - 3509.935256642997) / 6239.311524231554 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.071899700598802 < 28.88282627865961. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 68.50524939349717 (T) = (0 -19133.35991741407) / Math.Sqrt((6504836.867213558 / (299)) + (130729.14627446003 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.38972022691205405 = (31351.7844784556 - 19133.35991741407) / 31351.7844784556 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueTrue<Int32, Int32>.FrozenDictionaryOptimized(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.8224931940456797 < 3.345280633098302. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 193.86610454927103 (T) = (0 -2822.669465088934) / Math.Sqrt((1913.6599431970965 / (173)) + (56.78131153174369 / (13))) is greater than 1.972940542389176 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (173) + (13) - 2, .975) and 0.21246461627572558 = (3584.181134491329 - 2822.669465088934) / 3584.181134491329 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToStream(Mode%3a%20SourceGen).html>) 359.89 ns 330.03 ns 0.92 0.02 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToString(Mode%3a%20SourceGen).html>) 336.05 ns 295.22 ns 0.88 0.00 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToWriter(Mode%3a%20Reflection).html>) 279.02 ns 215.64 ns 0.77 0.02 True
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 622.41 ns 492.89 ns 0.79 0.23 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToString(Mode%3a%20Reflection).html>) 415.47 ns 359.01 ns 0.86 0.01 True
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 380.04 ns 309.75 ns 0.82 0.00 True
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToStream(Mode%3a%20Reflection).html>) 477.24 ns 414.23 ns 0.87 0.02 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeObjectProperty(Mode%3a%20Reflection).html>) 628.00 ns 523.72 ns 0.83 0.17 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 300.81 ns 256.70 ns 0.85 0.02 True
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToWriter(Mode%3a%20SourceGen).html>) 193.19 ns 163.35 ns 0.85 0.01 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 330.0285701661365 < 341.8935684206025. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.80904352577215 (T) = (0 -327.7385637893822) / Math.Sqrt((45.366255395733134 / (299)) + (20.24615848192114 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11572803794040355 = (370.6309572747642 - 327.7385637893822) / 370.6309572747642 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 295.2239504940776 < 320.323750319742. IsChangePoint: Marked as a change because one of 4/19/2023 10:34:24 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 118.97399527841706 (T) = (0 -295.5608569327439) / Math.Sqrt((28.7805188834503 / (299)) + (0.6343237313533845 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13292873261347857 = (340.872622643358 - 295.5608569327439) / 340.872622643358 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 215.63965550852177 < 269.1717330035874. IsChangePoint: Marked as a change because one of 3/20/2023 7:45:43 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 114.31282181736212 (T) = (0 -215.4347850855132) / Math.Sqrt((61.28941387953263 / (299)) + (2.1428847715112473 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24395984960511236 = (284.95151345201623 - 215.4347850855132) / 284.95151345201623 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 492.8894451280904 < 536.7363763431327. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.760642254583853 (T) = (0 -491.2794692242795) / Math.Sqrt((409.28704476620607 / (299)) + (215.6636915385285 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15804029492004001 = (583.4952269807535 - 491.2794692242795) / 583.4952269807535 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 359.00780852359105 < 394.65735566573954. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.164620162421116 (T) = (0 -355.9264619383723) / Math.Sqrt((41.190631474103135 / (299)) + (55.50476678001942 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15527638070616695 = (421.352562907993 - 355.9264619383723) / 421.352562907993 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 309.74576698754817 < 361.13332042503504. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 88.7527090914558 (T) = (0 -311.35014688930227) / Math.Sqrt((42.56717330649911 / (299)) + (6.941940846820407 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18991252362344985 = (384.34139024336486 - 311.35014688930227) / 384.34139024336486 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 414.2348520336367 < 447.70493404756814. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 26.99113829872437 (T) = (0 -407.2602258403914) / Math.Sqrt((99.0085657712501 / (299)) + (107.5614755803195 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16276917776050365 = (486.43721065000574 - 407.2602258403914) / 486.43721065000574 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 523.7167062126124 < 626.5505513810496. IsChangePoint: Marked as a change because one of 4/12/2023 6:21:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.443082552644379 (T) = (0 -543.8012088385262) / Math.Sqrt((372.8985275694821 / (299)) + (643.1183507050764 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14969685102173663 = (639.5380394533005 - 543.8012088385262) / 639.5380394533005 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 256.6984211246327 < 286.2880334878634. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 89.3848681803311 (T) = (0 -255.93243798980407) / Math.Sqrt((25.631366379761367 / (299)) + (2.307809966412531 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1519623238452838 = (301.7937117490894 - 255.93243798980407) / 301.7937117490894 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 163.34563878168836 < 183.5422619008326. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 111.37054283270089 (T) = (0 -163.5781749013016) / Math.Sqrt((8.209630232280947 / (299)) + (0.5243823542457152 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15057882024385205 = (192.57604919653835 - 163.5781749013016) / 192.57604919653835 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>) 10.04 μs 8.57 μs 0.85 0.01 True
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%20%201%2c%20%20%20%201%5d).html>) 214.77 μs 198.90 μs 0.93 0.18 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%201%2c%20%20%20%201%5d).html>) 113.34 μs 98.51 μs 0.87 0.01 True
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>) 15.23 μs 13.79 μs 0.91 0.01 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%201%2c%20%20%20%208%5d).html>) 73.63 μs 64.44 μs 0.88 0.01 False
[ReadLineAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange%3a%20%5b%20%20%200%2c%20%20%20%200%5d).html>) 244.78 μs 226.65 μs 0.93 0.33 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%209%2c%20%20%2032%5d).html>) 24.02 μs 21.23 μs 0.88 0.00 True

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 33, 128]) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.567522896264054 < 9.562088612222125. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 108.57312176001209 (T) = (0 -8514.432118944538) / Math.Sqrt((9432.829074465391 / (299)) + (2247.9300269295068 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15421804616966606 = (10066.935195749702 - 8514.432118944538) / 10066.935195749702 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 1]) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 198.9006353021978 < 204.08267811276906. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.808943594984072 (T) = (0 -200738.19644093758) / Math.Sqrt((28109466.40200305 / (299)) + (4247342.152632854 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08516155385065426 = (219424.74902084234 - 200738.19644093758) / 219424.74902084234 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 1]) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 98.51407837445574 < 107.38030005706135. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 58.16984162615375 (T) = (0 -98870.54665687414) / Math.Sqrt((1315013.2367502274 / (299)) + (695473.8270824306 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12400943915623155 = (112867.13701759573 - 98870.54665687414) / 112867.13701759573 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 33, 128]) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.786986926672508 < 14.078504315325967. IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.67097990159588 (T) = (0 -13739.22744255159) / Math.Sqrt((59342.70643251337 / (299)) + (13890.161852443895 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09532240015372419 = (15186.87700998254 - 13739.22744255159) / 15186.87700998254 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 8]) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 64.43858222538778 < 69.61530134229373. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 92.37063282637823 (T) = (0 -64338.911975237046) / Math.Sqrt((2279929.356466092 / (299)) + (41550.57962539685 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1299420501893201 = (73947.84679484494 - 64338.911975237046) / 73947.84679484494 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.StringReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 0, 0]) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 226.64935119047618 < 236.9521788155692. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 54.74021573575987 (T) = (0 -223680.9816930429) / Math.Sqrt((66410783.33601062 / (299)) + (1208707.3100761124 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12077885302220616 = (254408.10023953204 - 223680.9816930429) / 254408.10023953204 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32]) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 21.22916383989145 < 22.803160810879607. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.06282622027075 (T) = (0 -21162.879624792437) / Math.Sqrt((50614.62242652494 / (299)) + (35711.49158533398 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12319775435564344 = (24136.434104636635 - 21162.879624792437) / 24136.434104636635 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in GuardedDevirtualization.TwoClassInterface

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassInterface.Call(testInput%3a%20pB%20%3d%200.80).html>) 3.66 ns 1.26 ns 0.34 0.12 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassInterface.Call(testInput%3a%20pB%20%3d%200.60).html>) 4.89 ns 2.62 ns 0.54 0.13 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassInterface.Call(testInput%3a%20pB%20%3d%200.70).html>) 4.43 ns 2.12 ns 0.48 0.11 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassInterface.Call(testInput%3a%20pB%20%3d%200.20).html>) 5.31 ns 1.47 ns 0.28 0.10 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassInterface.Call(testInput%3a%20pB%20%3d%200.90).html>) 2.52 ns 0.95 ns 0.38 0.11 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassInterface.Call(testInput%3a%20pB%20%3d%200.50).html>) 5.37 ns 3.47 ns 0.65 0.17 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassInterface.Call(testInput%3a%20pB%20%3d%200.40).html>) 5.70 ns 2.83 ns 0.50 0.13 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassInterface.Call(testInput%3a%20pB%20%3d%200.30).html>) 5.76 ns 2.09 ns 0.36 0.11 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassInterface.Call(testInput%3a%20pB%20%3d%200.10).html>) 4.78 ns 1.05 ns 0.22 0.14 True

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.TwoClassInterface*'
### Payloads [Baseline]() [Compare]() ### Histogram #### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.80) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2601566186640631 < 3.479047754744919. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 127.98467272123787 (T) = (0 -1.2601961329203883) / Math.Sqrt((0.11857187169831 / (299)) + (4.426332469507626E-05 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6700868800902702 = (3.8197818057832955 - 1.2601961329203883) / 3.8197818057832955 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.60) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.616575291204006 < 4.639993543810139. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.091122572107485 (T) = (0 -2.6460648079998554) / Math.Sqrt((0.12117623691649726 / (299)) + (0.43652429291568134 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4769984922387594 = (5.0593827526933834 - 2.6460648079998554) / 5.0593827526933834 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.70) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.1182784311105167 < 4.208804083792953. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.59332490041951 (T) = (0 -1.7957501617547675) / Math.Sqrt((0.10836281273884601 / (299)) + (0.041863874454622425 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6133587020419166 = (4.644486171649074 - 1.7957501617547675) / 4.644486171649074 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.20) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.4669902976522333 < 5.049920296476382. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 295.0651808934359 (T) = (0 -1.4627411560460173) / Math.Sqrt((0.042555430130499235 / (299)) + (0.000506788694207746 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7309075256336449 = (5.435830784530129 - 1.4627411560460173) / 5.435830784530129 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.90) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0.9487565601752778 < 2.3946220222032406. IsChangePoint: Marked as a change because one of 3/9/2023 9:18:02 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 103.32874712238002 (T) = (0 -0.9487994191101365) / Math.Sqrt((0.08960627875138329 / (299)) + (1.3323795016208637E-06 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6534542607030088 = (2.737876451849871 - 0.9487994191101365) / 2.737876451849871 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.50) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.467759817197021 < 5.0959652423724915. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 25.28182364492617 (T) = (0 -3.121460097475414) / Math.Sqrt((0.09805411328578419 / (299)) + (0.12285008619505357 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.44471989510554144 = (5.62141533608287 - 3.121460097475414) / 5.62141533608287 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.40) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.8256961594589285 < 5.406377307500815. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.45042658429034 (T) = (0 -2.921677763154121) / Math.Sqrt((0.15202362557792418 / (299)) + (0.06287818863680819 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5091400387530005 = (5.952161499853805 - 2.921677763154121) / 5.952161499853805 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.30) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.0941326530612243 < 5.482384730802809. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.660583827747562 (T) = (0 -2.4160109999789223) / Math.Sqrt((0.05109110227663886 / (299)) + (0.4217280600195301 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5950670831467799 = (5.966447526059427 - 2.4160109999789223) / 5.966447526059427 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0458964968866191 < 4.5410210058165985. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 243.89885867085258 (T) = (0 -1.0410668213635117) / Math.Sqrt((0.07072696178391717 / (299)) + (0.00011727200670585458 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7859250316605917 = (4.863094594567156 - 1.0410668213635117) / 4.863094594567156 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(TreeRecord).DeserializeFromString(Mode%3a%20SourceGen).html>) 43.26 μs 37.74 μs 0.87 0.34 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(TreeRecord).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 43.94 μs 36.10 μs 0.82 0.32 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(TreeRecord).DeserializeFromStream(Mode%3a%20Reflection).html>) 38.75 μs 34.09 μs 0.88 0.41 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(TreeRecord).DeserializeFromStream(Mode%3a%20SourceGen).html>) 45.95 μs 33.77 μs 0.73 0.38 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(TreeRecord).DeserializeFromString(Mode%3a%20Reflection).html>) 42.99 μs 35.27 μs 0.82 0.44 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(TreeRecord).DeserializeFromReader(Mode%3a%20Reflection).html>) 46.76 μs 35.49 μs 0.76 0.40 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(TreeRecord).DeserializeFromReader(Mode%3a%20SourceGen).html>) 50.90 μs 39.32 μs 0.77 0.32 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(TreeRecord).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 39.99 μs 31.86 μs 0.80 0.45 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;TreeRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 37.74317899378634 < 41.676491963140734. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.930268918380053 (T) = (0 -35848.66800982589) / Math.Sqrt((3615763.8526705327 / (299)) + (2729339.3071498456 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15476764902337783 = (42412.796869883896 - 35848.66800982589) / 42412.796869883896 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 36.10342799642217 < 41.36732769419334. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.227200087902364 (T) = (0 -34536.96998983848) / Math.Sqrt((4358516.101856351 / (299)) + (5097675.644328819 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18419207053235953 = (42334.682885928494 - 34536.96998983848) / 42334.682885928494 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 34.091552676254224 < 37.97316322977554. IsChangePoint: Marked as a change because one of 4/18/2023 2:18:01 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.987595535305571 (T) = (0 -31977.573788304122) / Math.Sqrt((4790371.886399083 / (299)) + (5460925.022097493 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18494179559783117 = (39233.484940819806 - 31977.573788304122) / 39233.484940819806 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 33.77312562290478 < 43.632736088754285. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 4/17/2023 5:51:51 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 15.962537415897867 (T) = (0 -35503.18959353308) / Math.Sqrt((6346177.293025539 / (299)) + (2950705.3691903898 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18300281259645593 = (43455.706018234785 - 35503.18959353308) / 43455.706018234785 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 35.26698130728808 < 38.69913441454342. IsChangePoint: Marked as a change because one of 3/4/2023 1:07:20 AM, 4/17/2023 2:42:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.018391543644917 (T) = (0 -31388.848557318015) / Math.Sqrt((5099593.765391477 / (299)) + (5950896.749038026 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19477134734752424 = (38981.28618985564 - 31388.848557318015) / 38981.28618985564 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 35.487969763302715 < 47.15090764064968. IsChangePoint: Marked as a change because one of 2/18/2023 8:36:57 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.459588475345583 (T) = (0 -39743.37315704097) / Math.Sqrt((5245474.747783667 / (299)) + (4808833.232663036 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17410186278024223 = (48121.3982281521 - 39743.37315704097) / 48121.3982281521 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 39.32117058444008 < 50.70321155438282. IsChangePoint: Marked as a change because one of 2/20/2023 2:00:07 PM, 4/12/2023 6:21:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.18284141715499 (T) = (0 -42752.53686397461) / Math.Sqrt((5555592.425340626 / (299)) + (6144495.173626197 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17771151798300353 = (51992.13876753648 - 42752.53686397461) / 51992.13876753648 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<TreeRecord>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 31.85794264000724 < 39.01286156501428. IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.994274059522688 (T) = (0 -31742.033022166604) / Math.Sqrt((4684915.88960293 / (299)) + (5745590.590027223 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17559686989330808 = (38503.04767530254 - 31742.033022166604) / 38503.04767530254 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromString(Mode%3a%20SourceGen).html>) 1.58 μs 973.54 ns 0.61 0.23 True
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 1.37 μs 1.01 μs 0.73 0.10 True
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromStream(Mode%3a%20Reflection).html>) 1.76 μs 1.22 μs 0.69 0.21 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromReader(Mode%3a%20SourceGen).html>) 2.07 μs 1.59 μs 0.77 0.08 True
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromStream(Mode%3a%20SourceGen).html>) 1.80 μs 1.20 μs 0.67 0.21 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 1.36 μs 1.02 μs 0.75 0.16 True
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromReader(Mode%3a%20Reflection).html>) 2.05 μs 1.54 μs 0.75 0.16 True
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromString(Mode%3a%20Reflection).html>) 1.41 μs 958.46 ns 0.68 0.19 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 973.5393906247203 < 1.4140114234928889. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 25.79046235669044 (T) = (0 -1033.936137684883) / Math.Sqrt((1513.918822724952 / (299)) + (4179.947755557755 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.310719237621821 = (1500.0217532802785 - 1033.936137684883) / 1500.0217532802785 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0092226544844063 < 1.3163194981012531. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.71523796536502 (T) = (0 -984.589757519976) / Math.Sqrt((828.0747023506339 / (299)) + (2467.5504492580785 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2881187111432034 = (1383.081382994515 - 984.589757519976) / 1383.081382994515 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2235801585121824 < 1.704252485680928. IsChangePoint: Marked as a change because one of 2/12/2023 9:02:44 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 45.18121763332814 (T) = (0 -1271.6914457118992) / Math.Sqrt((2582.3305785225434 / (299)) + (1792.7681984047001 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30074214853200737 = (1818.630199206435 - 1271.6914457118992) / 1818.630199206435 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.590664625055064 < 1.9640297324195424. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.31179512213744 (T) = (0 -1544.6295201403182) / Math.Sqrt((1298.4178507081047 / (299)) + (3316.4134643267607 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2635174883050839 = (2097.306447352782 - 1544.6295201403182) / 2097.306447352782 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2035641009496394 < 1.7564627921258955. IsChangePoint: Marked as a change because one of 2/20/2023 12:02:23 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.64372032180625 (T) = (0 -1270.8614870175552) / Math.Sqrt((3761.2885309293174 / (299)) + (1598.0537515950266 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30822797802310287 = (1837.1102713662474 - 1270.8614870175552) / 1837.1102713662474 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0241472878270548 < 1.2805000285308803. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 35.97621188199977 (T) = (0 -955.4949038209761) / Math.Sqrt((1042.8620271284958 / (299)) + (1680.2303226016418 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3025489621014656 = (1369.981334747088 - 955.4949038209761) / 1369.981334747088 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.5394647164093047 < 1.9521362546869185. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.38044495220485 (T) = (0 -1541.6567079561553) / Math.Sqrt((2049.6276078569344 / (299)) + (3506.0720477403647 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2588685988724221 = (2080.13950779934 - 1541.6567079561553) / 2080.13950779934 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 958.4629389771371 < 1.3979746808471776. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.245531942821014 (T) = (0 -1031.3942985460262) / Math.Sqrt((1442.3485691203118 / (299)) + (2995.4672062679233 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3030898015682495 = (1479.9529420963586 - 1031.3942985460262) / 1479.9529420963586 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonFromStream(LoginViewModel).JsonNet.html>) 1.61 μs 1.19 μs 0.74 0.01 True
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_FromStream(LoginViewModel).SystemTextJsonSourceGen.html>) 593.02 ns 496.03 ns 0.84 0.01 True
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonFromStream(LoginViewModel).DataContractJsonSerializer.html>) 5.85 μs 4.55 μs 0.78 0.29 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_FromStream(LoginViewModel).SystemTextJsonReflection.html>) 605.44 ns 485.04 ns 0.80 0.01 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.JsonNet_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1939920918269415 < 1.5385645246000872. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 65.09968324227 (T) = (0 -1222.222681224922) / Math.Sqrt((1514.5892278880651 / (299)) + (604.1808538349429 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2766143486017195 = (1689.5865695737898 - 1222.222681224922) / 1689.5865695737898 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 496.0278724568158 < 567.5066746324644. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.85470270769026 (T) = (0 -473.9496687239512) / Math.Sqrt((131.26012463797719 / (299)) + (123.36855715975025 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20529094420432378 = (596.3813615404505 - 473.9496687239512) / 596.3813615404505 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.547163700525985 < 5.410056193652011. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.634833668818434 (T) = (0 -4824.5264900431785) / Math.Sqrt((43722.482269864835 / (299)) + (50647.63394839835 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17980094776398423 = (5882.141020390865 - 4824.5264900431785) / 5882.141020390865 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 485.03784078160965 < 593.0451392825881. IsChangePoint: Marked as a change because one of 3/2/2023 11:15:04 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 131.17547690847897 (T) = (0 -483.24364075257597) / Math.Sqrt((88.03160085236155 / (299)) + (8.974930733177898 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21221192495727173 = (613.418324117645 - 483.24364075257597) / 613.418324117645 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20NoneEscaped).html>) 6.01 ms 5.51 ms 0.92 0.01 False
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20OneEscaped).html>) 14.94 ms 14.04 ms 0.94 0.04 False
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20OneEscaped).html>) 15.04 ms 14.03 ms 0.93 0.02 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: False, Escaped: NoneEscaped) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.507746428571428 < 5.749350576986892. IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 27.68083348906593 (T) = (0 -5556800.5278333705) / Math.Sqrt((3911832168.9384494 / (299)) + (2760261218.843063 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06958542086113673 = (5972391.934116527 - 5556800.5278333705) / 5972391.934116527 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: OneEscaped) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.044598333333333 < 14.10691026488095. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.02547311939676 (T) = (0 -14042841.488564853) / Math.Sqrt((252779034988.78897 / (299)) + (17095715597.259644 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06506484248506154 = (15020123.455289438 - 14042841.488564853) / 15020123.455289438 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: OneEscaped) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.03210576923077 < 14.28036431845238. IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.01050151050173 (T) = (0 -14161812.334460411) / Math.Sqrt((156418773719.63376 / (299)) + (8266035972.330569 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06937518759830655 = (15217531.432363775 - 14161812.334460411) / 15217531.432363775 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromReader(Mode%3a%20SourceGen).html>) 2.10 μs 1.64 μs 0.78 0.01 True
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromReader(Mode%3a%20Reflection).html>) 2.08 μs 1.64 μs 0.79 0.01 True
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 1.39 μs 1.10 μs 0.79 0.00 True
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromString(Mode%3a%20Reflection).html>) 1.52 μs 1.17 μs 0.77 0.00 True
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromString(Mode%3a%20SourceGen).html>) 1.50 μs 1.16 μs 0.78 0.01 True
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 1.41 μs 1.08 μs 0.77 0.00 True
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromStream(Mode%3a%20SourceGen).html>) 1.92 μs 1.40 μs 0.73 0.08 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromStream(Mode%3a%20Reflection).html>) 1.82 μs 1.46 μs 0.80 0.06 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.6421121191656625 < 1.9942195932811646. IsChangePoint: Marked as a change because one of 2/20/2023 2:42:44 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 53.68615829846757 (T) = (0 -1677.874235977045) / Math.Sqrt((1119.8789107800164 / (299)) + (741.6605761022303 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19966908678652703 = (2096.475605621776 - 1677.874235977045) / 2096.475605621776 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.6449127240830064 < 2.00197866528792. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 68.49628210982075 (T) = (0 -1683.4767352014517) / Math.Sqrt((876.4041415473681 / (299)) + (448.7775356103443 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19935976129695174 = (2102.663161082816 - 1683.4767352014517) / 2102.663161082816 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0994584966519683 < 1.3270669935815622. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 86.62286606338348 (T) = (0 -1080.387348637877) / Math.Sqrt((649.6869994774717 / (299)) + (140.8615453989742 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22431162343426067 = (1392.8110582514505 - 1080.387348637877) / 1392.8110582514505 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1668311561874807 < 1.4403148922991196. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 141.54531604164788 (T) = (0 -1157.8365537055818) / Math.Sqrt((729.6848815315091 / (299)) + (41.29485457204288 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22464586805985645 = (1493.3002946774332 - 1157.8365537055818) / 1493.3002946774332 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1612542637685488 < 1.431548557841792. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 66.10340107095216 (T) = (0 -1159.5855243241867) / Math.Sqrt((923.6073208615526 / (299)) + (300.8555742166899 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22598669239778804 = (1498.1467539833716 - 1159.5855243241867) / 1498.1467539833716 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0843574685042108 < 1.3327906060991446. IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 104.65787824690358 (T) = (0 -1083.7355609381366) / Math.Sqrt((713.8205365116819 / (299)) + (86.94666849394996 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22536342500063766 = (1399.0245179670592 - 1083.7355609381366) / 1399.0245179670592 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.396914423147165 < 1.8250026119053497. IsChangePoint: Marked as a change because one of 2/6/2023 4:57:06 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.7995379102485 (T) = (0 -1416.3720010445563) / Math.Sqrt((2269.793318425571 / (299)) + (350.7930228486423 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23936007067425802 = (1862.0794760276106 - 1416.3720010445563) / 1862.0794760276106 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.4615890260081603 < 1.7608242971769767. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.44837021659727 (T) = (0 -1427.2536100836073) / Math.Sqrt((1810.6632268991257 / (299)) + (488.6753758143773 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22977441429638937 = (1853.0332367234898 - 1427.2536100836073) / 1853.0332367234898 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cw%2b%5c%5cs%2bHolmes%5c%5cs%2b%5c%5cw%2b%22%2c%20Options%3a%20None).html>) 13.66 ms 12.55 ms 0.92 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5b%5e%5c%5cn%5d*%22%2c%20Options%3a%20Compiled).html>) 1.15 ms 1.02 ms 0.88 0.04 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22.*%22%2c%20Options%3a%20Compiled).html>) 1.16 ms 1.02 ms 0.88 0.03 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5ba-zA-Z%5d%2bing%22%2c%20Options%3a%20None).html>) 20.48 ms 17.29 ms 0.84 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cw%2b%22%2c%20Options%3a%20NonBacktracking).html>) 19.62 ms 16.33 ms 0.83 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cp%7bL%7d%22%2c%20Options%3a%20NonBacktracking).html>) 47.80 ms 40.68 ms 0.85 0.03 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cp%7bLu%7d%22%2c%20Options%3a%20NonBacktracking).html>) 2.74 ms 2.42 ms 0.88 0.03 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%7cHolmes%7cWatson%7cIrene%7cAdler%7cJohn%7cBaker%22%2c%20Options%3a%20None).html>) 38.80 ms 32.21 ms 0.83 0.01 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22the%22%2c%20Options%3a%20NonBacktracking).html>) 1.15 ms 964.92 μs 0.84 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Holmes.%7b0%2c25%7dWatson%7cWatson.%7b0%2c25%7dHolmes%22%2c%20Options%3a%20None).html>) 551.84 μs 466.05 μs 0.84 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22.*%22%2c%20Options%3a%20None).html>) 2.06 ms 1.77 ms 0.86 0.10 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cHolmes%7cWatson%22%2c%20Options%3a%20None).html>) 504.50 μs 445.16 μs 0.88 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5b%5e%5c%5cn%5d*%22%2c%20Options%3a%20NonBacktracking).html>) 8.76 ms 7.82 ms 0.89 0.02 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sher%5ba-z%5d%2b%7cHol%5ba-z%5d%2b%22%2c%20Options%3a%20NonBacktracking).html>) 247.61 μs 224.04 μs 0.90 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sher%5ba-z%5d%2b%7cHol%5ba-z%5d%2b%22%2c%20Options%3a%20None).html>) 357.30 μs 312.15 μs 0.87 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22the%5c%5cs%2b%5c%5cw%2b%22%2c%20Options%3a%20NonBacktracking).html>) 1.60 ms 1.40 ms 0.88 0.02 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fs).*%22%2c%20Options%3a%20Compiled).html>) 82.71 ns 71.15 ns 0.86 0.01 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cw%2b%22%2c%20Options%3a%20Compiled).html>) 7.11 ms 6.59 ms 0.93 0.06 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22.*%22%2c%20Options%3a%20NonBacktracking).html>) 8.51 ms 7.50 ms 0.88 0.02 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cs%5ba-zA-Z%5d%7b0%2c12%7ding%5c%5cs%22%2c%20Options%3a%20None).html>) 23.13 ms 19.44 ms 0.84 0.01 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cw%2b%5c%5cs%2bHolmes%22%2c%20Options%3a%20None).html>) 13.66 ms 12.37 ms 0.91 0.01 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%7cHolmes%7cWatson%22%2c%20Options%3a%20None).html>) 13.12 ms 9.80 ms 0.75 0.38 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cHolmes%22%2c%20Options%3a%20None).html>) 332.96 μs 299.16 μs 0.90 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5ba-q%5d%5b%5eu-z%5d%7b13%7dx%22%2c%20Options%3a%20NonBacktracking).html>) 108.12 μs 101.78 μs 0.94 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cp%7bLl%7d%22%2c%20Options%3a%20None).html>) 34.58 ms 31.26 ms 0.90 0.01 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22The%22%2c%20Options%3a%20NonBacktracking).html>) 219.91 μs 202.66 μs 0.92 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cb%5c%5cw%2bn%5c%5cb%22%2c%20Options%3a%20None).html>) 38.79 ms 31.40 ms 0.81 0.34 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cp%7bLu%7d%22%2c%20Options%3a%20None).html>) 2.33 ms 2.18 ms 0.93 0.01 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cHolmes%22%2c%20Options%3a%20NonBacktracking).html>) 216.78 μs 198.60 μs 0.92 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cHolmes%7cWatson%7cIrene%7cAdler%7cJohn%7cBaker%22%2c%20Options%3a%20None).html>) 2.99 ms 2.64 ms 0.88 0.00 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22the%22%2c%20Options%3a%20None).html>) 847.36 μs 778.49 μs 0.92 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%7cStreet%22%2c%20Options%3a%20None).html>) 154.85 μs 142.65 μs 0.92 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sher%5ba-z%5d%2b%7cHol%5ba-z%5d%2b%22%2c%20Options%3a%20None).html>) 7.26 ms 6.50 ms 0.90 0.02 True
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5b%5e%5c%5cn%5d*%22%2c%20Options%3a%20None).html>) 2.20 ms 1.78 ms 0.81 0.10 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cp%7bL%7d%22%2c%20Options%3a%20None).html>) 35.84 ms 32.11 ms 0.90 0.02 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+\\s+Holmes\\s+\\w+", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.548442307692309 < 12.976834663461538. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 90.27023157760279 (T) = (0 -12526857.921245422) / Math.Sqrt((5504982838.544281 / (299)) + (2160278371.6046524 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08917393079680247 = (13753293.131150803 - 12526857.921245422) / 13753293.131150803 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\\n]*", Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0203556770833333 < 1.095153831959707. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 86.18268255585403 (T) = (0 -1023664.0897799116) / Math.Sqrt((487371774.7928982 / (299)) + (8331302.728341951 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11258624763457215 = (1153536.4276824698 - 1023664.0897799116) / 1153536.4276824698 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0216391666666667 < 1.09527951284913. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 80.30686119287815 (T) = (0 -1022519.0458965114) / Math.Sqrt((718742129.1183267 / (299)) + (3152884.796755826 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11328881329908576 = (1153159.0682879305 - 1022519.0458965114) / 1153159.0682879305 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.29061964285714 < 19.388858014880952. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 191.39466677156827 (T) = (0 -17224719.047971256) / Math.Sqrt((28523754588.817074 / (299)) + (2778026612.0587864 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16342740001142964 = (20589628.501108676 - 17224719.047971256) / 20589628.501108676 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+", Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 16.327830357142858 < 18.431022833333333. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 69.93847847228325 (T) = (0 -16014113.877148492) / Math.Sqrt((21917700159.20638 / (299)) + (26568149961.748478 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1673215114430037 = (19232049.461131644 - 16014113.877148492) / 19232049.461131644 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\p{L}", Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 40.68428583333333 < 45.59148420138889. IsChangePoint: Marked as a change because one of 5/11/2023 7:02:02 PM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.62237750521911 (T) = (0 -41052994.679487176) / Math.Sqrt((1748319535920.9934 / (299)) + (335707389750.6412 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1465839663253134 = (48104316.13608065 - 41052994.679487176) / 48104316.13608065 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\p{Lu}", Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.4175629251700683 < 2.592755487509385. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.092229452961572 (T) = (0 -2479846.907193586) / Math.Sqrt((6121088922.757281 / (299)) + (2228921784.570813 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1186262283811871 = (2813615.5023525027 - 2479846.907193586) / 2813615.5023525027 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 32.21177767857143 < 37.300809763888886. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.327159752679336 (T) = (0 -32863274.248027615) / Math.Sqrt((128477762989.18907 / (299)) + (1336227194492.6956 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16577639618405118 = (39393843.686156474 - 32863274.248027615) / 39393843.686156474 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 964.9217022441159 < 1.0825318953706586. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 100.61283042138696 (T) = (0 -955297.7803622282) / Math.Sqrt((97254387.00695857 / (299)) + (44304398.55514542 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16908899122490129 = (1149699.270166725 - 955297.7803622282) / 1149699.270166725 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 466.0473840497739 < 529.9823180675287. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 52.21168567700903 (T) = (0 -467665.5259043056) / Math.Sqrt((11663493.84319902 / (299)) + (35114490.53220622 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15598036634340298 = (554093.1836836664 - 467665.5259043056) / 554093.1836836664 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7716294599018 < 1.9507520677083332. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 90.42929856120321 (T) = (0 -1776631.5591319853) / Math.Sqrt((1863444915.872323 / (299)) + (57134564.23805082 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14231467371763468 = (2071425.8536202197 - 1776631.5591319853) / 2071425.8536202197 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 445.1643229166667 < 479.0485267857142. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 184.82583826302647 (T) = (0 -444510.64360678114) / Math.Sqrt((6029003.3453466445 / (299)) + (1042522.6687762585 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11639031924353 = (503062.21546399273 - 444510.64360678114) / 503062.21546399273 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\\n]*", Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.823822332506204 < 8.16195726364153. IsChangePoint: Marked as a change because one of 4/18/2023 4:36:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.720120829426303 (T) = (0 -7648429.671573774) / Math.Sqrt((118212463030.0246 / (299)) + (13202389458.421074 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09638827668444773 = (8464287.784481132 - 7648429.671573774) / 8464287.784481132 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 224.04270254110617 < 234.7739688680222. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 45.27623815170793 (T) = (0 -225383.3157773917) / Math.Sqrt((4014438.8267848613 / (299)) + (3061098.969116606 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09109136474195503 = (247971.36591556744 - 225383.3157773917) / 247971.36591556744 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 312.14679807692306 < 336.3711585385101. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 132.5594053103833 (T) = (0 -313563.7792941881) / Math.Sqrt((2463127.4909538585 / (299)) + (1139800.3278504391 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11576923542135571 = (354617.58610446926 - 313563.7792941881) / 354617.58610446926 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\\s+\\w+", Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.3986054019014695 < 1.5113704763329312. IsChangePoint: Marked as a change because one of 4/23/2023 8:38:14 PM, 4/24/2023 7:54:29 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 46.62133623194508 (T) = (0 -1367993.7305147424) / Math.Sqrt((2059587691.7766473 / (299)) + (215867827.90266 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.141768193492409 = (1593967.6438717989 - 1367993.7305147424) / 1593967.6438717989 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 71.14746775011086 < 78.71471195854889. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 63.19941250407938 (T) = (0 -70.81049252067098) / Math.Sqrt((1.1509968225983307 / (299)) + (0.43530319358608766 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14708704615603346 = (83.02194520734783 - 70.81049252067098) / 83.02194520734783 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+", Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.593228542510121 < 6.77245573977591. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.36490414807459 (T) = (0 -6594763.418073156) / Math.Sqrt((19122477992.87705 / (299)) + (8521297439.400902 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07994860050225067 = (7167820.647491215 - 6594763.418073156) / 7167820.647491215 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.496916105769231 < 8.104926023706897. IsChangePoint: Marked as a change because one of 4/20/2023 12:14:19 AM, 4/20/2023 7:02:40 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 15.287152424671211 (T) = (0 -7624500.2511524325) / Math.Sqrt((84968681803.88983 / (299)) + (33819729308.982555 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09723347057655853 = (8445705.509288073 - 7624500.2511524325) / 8445705.509288073 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\s[a-zA-Z]{0,12}ing\\s", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.43856153846154 < 21.94501116815476. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.50622423119862 (T) = (0 -19332108.537751697) / Math.Sqrt((8096243026.914873 / (299)) + (159706546257.38824 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1653138355722737 = (23160930.85238341 - 19332108.537751697) / 23160930.85238341 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+\\s+Holmes", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.373282692307692 < 12.972788028846153. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 80.38168475679909 (T) = (0 -12451973.513665821) / Math.Sqrt((9108037484.11414 / (299)) + (2920741998.975069 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09347256179083266 = (13735903.612873016 - 12451973.513665821) / 13735903.612873016 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.801765 < 12.3623405. IsChangePoint: Marked as a change because one of 4/9/2023 9:22:58 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.15983897374795 (T) = (0 -9871103.289733209) / Math.Sqrt((680655842467.7516 / (299)) + (21157932063.581047 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17778293548758964 = (12005471.201922758 - 9871103.289733209) / 12005471.201922758 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 299.15506289308183 < 317.89436480496454. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 204.19344330731974 (T) = (0 -298869.0493902031) / Math.Sqrt((1430411.5804447813 / (299)) + (323115.25975177326 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10524398598308342 = (334022.95677059575 - 298869.0493902031) / 334022.95677059575 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 101.77743019348858 < 103.13676026364496. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 54.69710036718165 (T) = (0 -101605.51605241164) / Math.Sqrt((321706.0468965951 / (299)) + (234097.1136553248 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06921858459420839 = (109161.52210464452 - 101605.51605241164) / 109161.52210464452 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\p{Ll}", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 31.26204071428572 < 33.601321875000004. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.174198910456543 (T) = (0 -31680714.491758242) / Math.Sqrt((151172393284.32483 / (299)) + (459331680514.42377 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10279896738107806 = (35310608.59268353 - 31680714.491758242) / 35310608.59268353 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 202.65917735042734 < 208.11453668113404. IsChangePoint: Marked as a change because one of 3/16/2023 1:49:47 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 82.0505157619991 (T) = (0 -202706.0107683502) / Math.Sqrt((17398747.669959277 / (299)) + (133794.56587448154 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09578023359213736 = (224177.81417633427 - 202706.0107683502) / 224177.81417633427 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\b\\w+n\\b", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 31.402704166666666 < 36.944407195512824. IsChangePoint: Marked as a change because one of 3/30/2023 2:02:58 PM, 3/31/2023 7:02:21 PM, 4/9/2023 9:22:58 PM, 4/11/2023 5:52:15 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.608099288774167 (T) = (0 -31727637.311214425) / Math.Sqrt((24824607187389.016 / (299)) + (308280436243.1788 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22755284496098957 = (41074184.95134739 - 31727637.311214425) / 41074184.95134739 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\p{Lu}", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.1760005398110662 < 2.2176163243419866. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.90840104724027 (T) = (0 -2177465.8603183255) / Math.Sqrt((4338355052.912948 / (299)) + (39080437.24412605 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08752966257848585 = (2386341.52916299 - 2177465.8603183255) / 2386341.52916299 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: NonBacktracking) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 198.59606036536934 < 204.5186758028057. IsChangePoint: Marked as a change because one of 4/27/2023 1:05:59 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.407842207112704 (T) = (0 -198992.31368805614) / Math.Sqrt((7287447.932540953 / (299)) + (4487936.686465035 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07985860059060679 = (216262.7546329101 - 198992.31368805614) / 216262.7546329101 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.64313594224924 < 2.891515761119714. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 78.30423416039147 (T) = (0 -2649694.3945704023) / Math.Sqrt((5860660345.00023 / (299)) + (94585489.69575916 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13285263142699968 = (3055644.8541506925 - 2649694.3945704023) / 3055644.8541506925 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 778.486011904762 < 802.3178116414834. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 95.83188143592801 (T) = (0 -772545.6401660761) / Math.Sqrt((47718494.08553871 / (299)) + (7491302.686706834 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09617539741648325 = (854751.7272242985 - 772545.6401660761) / 854751.7272242985 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 142.65114109848486 < 147.63192527846533. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.264268993757774 (T) = (0 -142576.72878113756) / Math.Sqrt((424713.4690369469 / (299)) + (866700.2021543357 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0811626836105222 = (155170.80797434872 - 142576.72878113756) / 155170.80797434872 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.502445748987854 < 6.902236575630252. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 65.20394844445313 (T) = (0 -6469936.1287606815) / Math.Sqrt((7224784120.531645 / (299)) + (1775050537.8031628 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11328521450965556 = (7296524.468330447 - 6469936.1287606815) / 7296524.468330447 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\\n]*", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7805371428571433 < 2.0771311614583334. IsChangePoint: Marked as a change because one of 5/5/2023 8:30:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 90.4047148628442 (T) = (0 -1776746.238256195) / Math.Sqrt((1966490542.4345348 / (299)) + (55472343.27794965 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1435101296885088 = (2074450.9653220088 - 1776746.238256195) / 2074450.9653220088 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\p{L}", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 32.10789928571428 < 34.5311498125. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.10994118552757 (T) = (0 -32426098.39285715) / Math.Sqrt((195900513063.34424 / (299)) + (415033804930.11237 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10515315982054287 = (36236478.62058076 - 32426098.39285715) / 36236478.62058076 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 208.38 ns 175.18 ns 0.84 0.04 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToWriter(Mode%3a%20SourceGen).html>) 116.34 ns 96.87 ns 0.83 0.00 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToStream(Mode%3a%20Reflection).html>) 391.43 ns 299.53 ns 0.77 0.10 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 278.03 ns 220.51 ns 0.79 0.12 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToString(Mode%3a%20SourceGen).html>) 236.09 ns 200.50 ns 0.85 0.00 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToWriter(Mode%3a%20Reflection).html>) 202.07 ns 138.33 ns 0.68 0.12 True
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToStream(Mode%3a%20SourceGen).html>) 288.87 ns 248.71 ns 0.86 0.02 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToString(Mode%3a%20Reflection).html>) 304.49 ns 269.79 ns 0.89 0.07 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 493.19 ns 417.80 ns 0.85 0.24 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeObjectProperty(Mode%3a%20Reflection).html>) 561.33 ns 492.83 ns 0.88 0.30 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 175.17516854072178 < 195.79943682575572. IsChangePoint: Marked as a change because one of 3/20/2023 11:25:43 AM, 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.0480050599476 (T) = (0 -176.47731250474607) / Math.Sqrt((37.73799575632761 / (299)) + (4.470250896105456 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1346400990222319 = (203.93516305221073 - 176.47731250474607) / 203.93516305221073 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 96.87338666017588 < 110.37005359718088. IsChangePoint: Marked as a change because one of 3/20/2023 7:45:43 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 59.424373083146435 (T) = (0 -96.19549787309202) / Math.Sqrt((11.544185062663944 / (299)) + (1.1828820290880786 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18192962708623203 = (117.58829198332538 - 96.19549787309202) / 117.58829198332538 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 299.5330716402728 < 354.7899039220006. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.5220562692786 (T) = (0 -307.9182262677258) / Math.Sqrt((70.42769248080657 / (299)) + (153.02116861616602 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1876101845379135 = (379.02767908603363 - 307.9182262677258) / 379.02767908603363 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 220.5064137659444 < 262.80978991929555. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 80.06995518497853 (T) = (0 -217.8617081765931) / Math.Sqrt((88.04428376487041 / (299)) + (4.49378189170619 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22723407426590833 = (281.9245788686071 - 217.8617081765931) / 281.9245788686071 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 200.49799527969893 < 223.09544507782314. IsChangePoint: Marked as a change because one of 3/20/2023 7:45:43 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.92506681368473 (T) = (0 -204.5418518873904) / Math.Sqrt((66.35861388082797 / (299)) + (2.6379467334594 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10528006595799327 = (228.60991926640952 - 204.5418518873904) / 228.60991926640952 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 138.33014519538682 < 192.05598281501713. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 100.92027926665459 (T) = (0 -137.61514933528898) / Math.Sqrt((63.85911917957992 / (299)) + (2.7900618726213113 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3242699117387168 = (203.65402063031652 - 137.61514933528898) / 203.65402063031652 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 248.71339650655815 < 271.1536374656183. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.108639413940427 (T) = (0 -245.085489131962) / Math.Sqrt((44.00253776264195 / (299)) + (36.09872183208198 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1639597937079231 = (293.1503620130197 - 245.085489131962) / 293.1503620130197 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 269.7921097617389 < 284.37437575676927. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/11/2023 2:19:19 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.799218951422628 (T) = (0 -259.6021020091753) / Math.Sqrt((42.599118653746935 / (299)) + (54.56967692009161 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13710400670222894 = (300.8498173888159 - 259.6021020091753) / 300.8498173888159 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 417.7955601218859 < 468.8170262671917. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 4/11/2023 2:19:19 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.061225677280506 (T) = (0 -429.23560884544975) / Math.Sqrt((456.7085089060698 / (299)) + (514.8922023656577 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11924729636564518 = (487.35088416333406 - 429.23560884544975) / 487.35088416333406 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 492.82509478661365 < 541.6804434334266. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.600954600822465 (T) = (0 -454.78203480701393) / Math.Sqrt((811.8858949279221 / (299)) + (505.92752411130004 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17160614450922504 = (548.9925254667445 - 454.78203480701393) / 548.9925254667445 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 367.77 μs 307.82 μs 0.84 0.26 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromString(Mode%3a%20SourceGen).html>) 376.30 μs 286.25 μs 0.76 0.30 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromStream(Mode%3a%20SourceGen).html>) 399.78 μs 309.64 μs 0.77 0.26 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromString(Mode%3a%20Reflection).html>) 384.06 μs 316.46 μs 0.82 0.30 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromReader(Mode%3a%20SourceGen).html>) 487.61 μs 400.87 μs 0.82 0.15 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 368.45 μs 283.37 μs 0.77 0.30 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromStream(Mode%3a%20Reflection).html>) 411.09 μs 331.61 μs 0.81 0.33 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromReader(Mode%3a%20Reflection).html>) 488.90 μs 410.25 μs 0.84 0.19 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 307.8155637254902 < 355.51526601060993. IsChangePoint: Marked as a change because one of 2/8/2023 4:01:29 AM, 2/24/2023 9:01:57 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 27.776238346550233 (T) = (0 -294704.2121927362) / Math.Sqrt((328479198.5491342 / (299)) + (60337357.27436656 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1842118952070206 = (361250.93080086354 - 294704.2121927362) / 361250.93080086354 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 286.25171656361863 < 360.03730612338313. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.17681832647139 (T) = (0 -297747.7591122398) / Math.Sqrt((215945018.7252685 / (299)) + (111173541.02742243 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1982539465378278 = (371374.14998986095 - 297747.7591122398) / 371374.14998986095 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 309.6421458492003 < 382.44119147181965. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.01120840270131 (T) = (0 -320645.9532725577) / Math.Sqrt((240240332.78823054 / (299)) + (145086482.84152237 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19886456614805303 = (400239.38490756403 - 320645.9532725577) / 400239.38490756403 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 316.4582998324958 < 363.5519951875143. IsChangePoint: Marked as a change because one of 2/8/2023 4:45:44 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.324424810856456 (T) = (0 -304593.46128136606) / Math.Sqrt((224949282.1721747 / (299)) + (138068683.33113888 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1762466399873036 = (369762.9364166374 - 304593.46128136606) / 369762.9364166374 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 400.8668373333333 < 461.4720828738846. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.89867342264346 (T) = (0 -403179.7663453043) / Math.Sqrt((128596834.65695223 / (299)) + (47273081.05018538 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.169821063133701 = (485654.05413343647 - 403179.7663453043) / 485654.05413343647 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 283.3718778280543 < 340.84165944918044. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.66959982965769 (T) = (0 -288321.20694514026) / Math.Sqrt((176638398.96223742 / (299)) + (130130727.5184409 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2038205455927433 = (362130.93084622605 - 288321.20694514026) / 362130.93084622605 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 331.61080876795154 < 386.54372951969. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 25.24445822861521 (T) = (0 -315362.80124336405) / Math.Sqrt((678935683.1013927 / (299)) + (122714442.40872075 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21502722911001557 = (401749.9879464772 - 315362.80124336405) / 401749.9879464772 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 410.2460797342193 < 464.6901547419812. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.84808133805019 (T) = (0 -403498.9487973499) / Math.Sqrt((251886761.19634452 / (299)) + (43685563.7815871 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16484210393042278 = (483140.91346833686 - 403498.9487973499) / 483140.91346833686 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractSerializerBinaryXml - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Xml_ToStream(SimpleStructWithProperties).DataContractSerializerBinaryXml.html>) 590.74 ns 417.31 ns 0.71 0.18 False
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlToStream(SimpleStructWithProperties).DataContractSerializer.html>) 1.04 μs 793.12 ns 0.76 0.24 False
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlToStream(SimpleStructWithProperties).XmlSerializer.html>) 2.01 μs 1.79 μs 0.89 0.18 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>.DataContractSerializer_BinaryXml_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 417.3097324297938 < 570.3662071056583. IsChangePoint: Marked as a change because one of 4/4/2023 3:37:20 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.729500592411334 (T) = (0 -419.14956682626973) / Math.Sqrt((1528.9042400675494 / (216)) + (431.29898964701016 / (13))) is greater than 1.9704695126513152 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (216) + (13) - 2, .975) and 0.36350879758182697 = (658.531595148254 - 419.14956682626973) / 658.531595148254 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>.DataContractSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 793.123780228961 < 0.9822453466077585. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.935344240935606 (T) = (0 -831.4088829627109) / Math.Sqrt((1120.4391207389776 / (299)) + (1781.136589033209 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23839444207410224 = (1091.652856666265 - 831.4088829627109) / 1091.652856666265 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>.XmlSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.792151145096249 < 1.8888225590451655. IsChangePoint: Marked as a change because one of 2/26/2023 7:54:25 AM, 4/26/2023 5:34:01 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.42953673919769 (T) = (0 -1765.286544028819) / Math.Sqrt((3269.556259616771 / (299)) + (1383.0440600897239 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12569120052930216 = (2019.0652834530713 - 1765.286544028819) / 2019.0652834530713 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlFromStream(IndexViewModel).DataContractSerializer.html>) 184.80 μs 162.75 μs 0.88 0.00 True
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlFromStream(IndexViewModel).XmlSerializer.html>) 96.26 μs 87.62 μs 0.91 0.23 False
[DataContractSerializerBinaryXml - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Xml_FromStream(IndexViewModel).DataContractSerializerBinaryXml.html>) 67.60 μs 59.90 μs 0.89 0.01 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.DataContractSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 162.7499210990439 < 176.56454235010375. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.8326561863947 (T) = (0 -164529.74894037645) / Math.Sqrt((3340673.8840551428 / (299)) + (2401045.856911512 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11820112627237499 = (186584.2130698812 - 164529.74894037645) / 186584.2130698812 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.XmlSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 87.62082312448958 < 95.69445147991458. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.276544452936836 (T) = (0 -85138.37103689149) / Math.Sqrt((7571779.046475326 / (299)) + (6087434.409332231 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13723873912464057 = (98681.26316950058 - 85138.37103689149) / 98681.26316950058 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Xml_FromStream<IndexViewModel>.DataContractSerializer_BinaryXml_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 59.90409080160796 < 64.02315406957425. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.22800695775476 (T) = (0 -58183.97170391119) / Math.Sqrt((880419.6488078785 / (214)) + (655797.1463869109 / (13))) is greater than 1.9705633902925863 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (214) + (13) - 2, .975) and 0.14200709216615573 = (67814.04738042297 - 58183.97170391119) / 67814.04738042297 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromString(Mode%3a%20Reflection).html>) 419.60 ns 320.98 ns 0.76 0.08 True
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 352.49 ns 260.31 ns 0.74 0.09 True
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromStream(Mode%3a%20SourceGen).html>) 669.77 ns 529.52 ns 0.79 0.06 True
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromReader(Mode%3a%20Reflection).html>) 576.85 ns 457.31 ns 0.79 0.16 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromString(Mode%3a%20SourceGen).html>) 440.18 ns 314.00 ns 0.71 0.09 True
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromStream(Mode%3a%20Reflection).html>) 659.68 ns 529.57 ns 0.80 0.11 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromReader(Mode%3a%20SourceGen).html>) 593.73 ns 440.17 ns 0.74 0.15 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 369.06 ns 259.36 ns 0.70 0.05 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 320.9831061344434 < 398.6599526170209. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 109.97411556513428 (T) = (0 -313.00581812177495) / Math.Sqrt((76.20130389440656 / (299)) + (11.390441140150228 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27201816531842343 = (429.9637754816856 - 313.00581812177495) / 429.9637754816856 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 260.3117475922953 < 334.9395795259885. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 145.07199016713102 (T) = (0 -260.05471987753515) / Math.Sqrt((42.20341340499717 / (299)) + (3.741968325267783 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2676026092880317 = (355.0732473592434 - 260.05471987753515) / 355.0732473592434 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 529.5220869841994 < 648.2788116168645. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 66.83412696924147 (T) = (0 -529.7092062281725) / Math.Sqrt((279.72721312818595 / (299)) + (60.70069569547771 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2300019353745955 = (687.9357631708726 - 529.7092062281725) / 687.9357631708726 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 457.3118523505649 < 550.9497845090968. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.6223352570427 (T) = (0 -468.8092857974046) / Math.Sqrt((225.30426511837928 / (299)) + (441.2431209103992 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20578551155779662 = (590.2804502055125 - 468.8092857974046) / 590.2804502055125 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 314.00080273981126 < 421.41372145006846. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 119.84113636371772 (T) = (0 -312.9950882014774) / Math.Sqrt((97.3212970109807 / (299)) + (11.941711815573429 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2992605913613344 = (446.6640299416534 - 312.9950882014774) / 446.6640299416534 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 529.5690666779161 < 624.6952398315499. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.26897987241511 (T) = (0 -530.3054243760954) / Math.Sqrt((445.38086919233564 / (299)) + (92.70565251368207 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21088873353249088 = (672.0287073710539 - 530.3054243760954) / 672.0287073710539 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 440.16589732715687 < 565.4730382552841. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.54308496016799 (T) = (0 -461.4607216597069) / Math.Sqrt((309.998978986672 / (299)) + (200.781049594444 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24327889887113274 = (609.8161145120778 - 461.4607216597069) / 609.8161145120778 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 259.3635400793718 < 350.09349416093505. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 100.58450352167486 (T) = (0 -259.8288444389799) / Math.Sqrt((67.31203965736506 / (299)) + (12.61370968063487 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2973851813150642 = (369.80268210866114 - 259.8288444389799) / 369.80268210866114 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseGeneric_Flags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.TryParseGeneric_Flags(text%3a%20%22Red%22).html>) 18.64 ns 16.72 ns 0.90 0.02 False
[StringFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.StringFormat(value%3a%20Red%2c%20Green).html>) 304.11 ns 284.48 ns 0.94 0.11 False
[StringFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.StringFormat(value%3a%20Red).html>) 274.49 ns 250.31 ns 0.91 0.12 False
[ToString_Flags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.ToString_Flags(value%3a%2036).html>) 31.74 ns 27.86 ns 0.88 0.19 False
[StringFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.StringFormat(value%3a%2032).html>) 282.35 ns 248.82 ns 0.88 0.11 False
[ToString_Format_NonFlags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.ToString_Format_NonFlags(value%3a%207%2c%20format%3a%20%22G%22).html>) 20.45 ns 18.95 ns 0.93 0.27 False
[ToString_Flags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.ToString_Flags(value%3a%2032).html>) 30.47 ns 26.76 ns 0.88 0.20 False
[GetValuesAsUnderlyingType_NonGeneric - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.GetValuesAsUnderlyingType_NonGeneric.html>) 26.77 ns 23.67 ns 0.88 0.12 False
[ToString_NonFlags_Small - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.ToString_NonFlags_Small(value%3a%20-1).html>) 35.28 ns 29.94 ns 0.85 0.13 False
[ToString_NonFlags_Large - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.ToString_NonFlags_Large(value%3a%2042).html>) 21.02 ns 18.13 ns 0.86 0.36 False
[GetName_NonGeneric_Flags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.GetName_NonGeneric_Flags.html>) 49.66 ns 39.34 ns 0.79 0.21 False
[ToString_Format_NonFlags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.ToString_Format_NonFlags(value%3a%20Tuesday%2c%20format%3a%20%22d%22).html>) 18.72 ns 15.41 ns 0.82 0.32 False
[GetValuesAsUnderlyingType_Generic - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.GetValuesAsUnderlyingType_Generic.html>) 22.68 ns 20.49 ns 0.90 0.29 False
[ToString_Format_NonFlags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.ToString_Format_NonFlags(value%3a%20Saturday%2c%20format%3a%20%22D%22).html>) 18.72 ns 15.02 ns 0.80 0.28 False
[GetValues_Generic - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Enum.GetValues_Generic.html>) 49.02 ns 40.57 ns 0.83 0.12 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 16.718818491461274 < 17.552338060606182. IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 33.54947593334099 (T) = (0 -17.086534089663402) / Math.Sqrt((0.7338006341111172 / (299)) + (0.02555894146482667 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11546972568964409 = (19.317070976440355 - 17.086534089663402) / 19.317070976440355 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.StringFormat(value: Red, Green) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 284.47583082273303 < 289.2746540858191. IsChangePoint: Marked as a change because one of 3/2/2023 11:15:04 AM, 4/19/2023 6:20:25 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.28196313268598 (T) = (0 -285.299240073325) / Math.Sqrt((73.89718317786163 / (299)) + (1.9198557234692462 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05863685120120583 = (303.07032991186753 - 285.299240073325) / 303.07032991186753 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.StringFormat(value: Red) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 250.31371185591055 < 255.46987168721316. IsChangePoint: Marked as a change because one of 3/5/2023 5:07:43 AM, 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.81825181519578 (T) = (0 -249.68837282532934) / Math.Sqrt((80.58919282749868 / (299)) + (2.140070404744496 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06146703152924486 = (266.0411314396034 - 249.68837282532934) / 266.0411314396034 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.ToString_Flags(value: 36) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.85783678631504 < 30.23570482112012. IsChangePoint: Marked as a change because one of 4/26/2023 2:24:59 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.107003511728117 (T) = (0 -27.949037901951595) / Math.Sqrt((0.9942156692813955 / (299)) + (0.981924738971374 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10039337255550582 = (31.068065807103064 - 27.949037901951595) / 31.068065807103064 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.StringFormat(value: 32) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 248.8164898276915 < 268.54814970758895. IsChangePoint: Marked as a change because one of 3/2/2023 11:15:04 AM, 4/26/2023 8:38:54 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.9265352249771 (T) = (0 -249.45947623682454) / Math.Sqrt((103.28979421277818 / (299)) + (2.729740493484666 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08955967580046473 = (273.9987120585315 - 249.45947623682454) / 273.9987120585315 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.ToString_Format_NonFlags(value: 7, format: "G") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.95206277782965 < 19.34450387030625. IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.111734913085286 (T) = (0 -18.53739576756959) / Math.Sqrt((0.6358478487819512 / (299)) + (0.06505555999905582 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11708685954388744 = (20.995718512008075 - 18.53739576756959) / 20.995718512008075 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.ToString_Flags(value: 32) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 26.75508867861176 < 29.06866272410756. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.020277841557654 (T) = (0 -27.232199185288103) / Math.Sqrt((1.0785127127840726 / (299)) + (0.5570312819542929 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09342447883397077 = (30.03853352477717 - 27.232199185288103) / 30.03853352477717 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.GetValuesAsUnderlyingType_NonGeneric ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.667445757179866 < 25.426623840651867. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.634498330504739 (T) = (0 -24.00823069628502) / Math.Sqrt((0.4510164090615906 / (299)) + (0.40443391862920736 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08679588713458672 = (26.290103557410628 - 24.00823069628502) / 26.290103557410628 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.ToString_NonFlags_Small(value: -1) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 29.93745825742134 < 33.44999462445478. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.537088736963483 (T) = (0 -29.301179137762627) / Math.Sqrt((1.2854499637410284 / (299)) + (1.0043549603945325 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2176101034765986 = (37.450865953106316 - 29.301179137762627) / 37.450865953106316 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.ToString_NonFlags_Large(value: 42) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.12534614251164 < 20.252244316374547. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 8.989408818098319 (T) = (0 -17.989311050754914) / Math.Sqrt((1.1418185023218816 / (299)) + (0.6439309267820892 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10347898908153484 = (20.06568817871349 - 17.989311050754914) / 20.06568817871349 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.GetName_NonGeneric_Flags ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 39.33753498888948 < 47.533038437857265. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.780652114833664 (T) = (0 -39.79535820060656) / Math.Sqrt((2.9432329118261538 / (299)) + (0.6130088533384493 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19267240762756854 = (49.292701719339235 - 39.79535820060656) / 49.292701719339235 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.ToString_Format_NonFlags(value: Tuesday, format: "d") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.411968624589816 < 17.46434089383054. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 46.22276890736641 (T) = (0 -15.345824572964098) / Math.Sqrt((0.6705488275216464 / (299)) + (0.011973411211215623 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14487427568052688 = (17.945693991578402 - 15.345824572964098) / 17.945693991578402 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.GetValuesAsUnderlyingType_Generic ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.4874228985064 < 21.418135904656378. IsChangePoint: Marked as a change because one of 2/15/2023 9:41:15 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 6.338034721711141 (T) = (0 -20.789530462473856) / Math.Sqrt((0.8995898155520933 / (299)) + (0.9691251414871312 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07825806058669842 = (22.55461054067542 - 20.789530462473856) / 22.55461054067542 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.ToString_Format_NonFlags(value: Saturday, format: "D") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.017575574308523 < 17.777062351095637. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.088387436130493 (T) = (0 -15.347324954886991) / Math.Sqrt((0.6252768679611268 / (299)) + (0.1519325405554638 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14452334945715709 = (17.940086319303212 - 15.347324954886991) / 17.940086319303212 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.GetValues_Generic ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 40.57239974831585 < 46.99075703197366. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/25/2023 6:46:56 PM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.459678460091688 (T) = (0 -41.124866457865046) / Math.Sqrt((10.15085173836486 / (299)) + (3.175942259716783 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.20544973168188635 = (51.758671663301115 - 41.124866457865046) / 51.758671663301115 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_SByte.ToString(value%3a%20-128).html>) 19.30 ns 16.01 ns 0.83 0.07 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_SByte.ToString(value%3a%20127).html>) 3.51 ns 1.11 ns 0.32 0.55 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_SByte.ToString(value: -128) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 16.013237288370497 < 18.440822401703286. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 60.9779691726554 (T) = (0 -15.929023330710423) / Math.Sqrt((0.9017797114819912 / (299)) + (0.1276184197955144 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3024820910961983 = (22.836723082485438 - 15.929023330710423) / 22.836723082485438 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_SByte.ToString(value: 127) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1068619015616465 < 3.3329046863383005. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.705430154802922 (T) = (0 -1.3584202095744566) / Math.Sqrt((0.23721895873730806 / (299)) + (0.033906686672961435 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.47061293772061175 = (2.5660245713703134 - 1.3584202095744566) / 2.5660245713703134 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20ShortSingleLine).html>) 102.84 ns 91.22 ns 0.89 0.02 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20LongMultiLine).html>) 18.70 μs 17.15 μs 0.92 0.00 True
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortMultiLine).html>) 92.44 ns 74.49 ns 0.81 0.04 True
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortSingleLine).html>) 90.22 ns 78.55 ns 0.87 0.09 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortSingleLine).html>) 90.34 ns 74.83 ns 0.83 0.02 True
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20LongMultiLine).html>) 18.89 μs 17.24 μs 0.91 0.00 True
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20ShortMultiLine).html>) 103.77 ns 89.83 ns 0.87 0.04 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortMultiLine).html>) 92.54 ns 77.69 ns 0.84 0.04 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20ShortMultiLine).html>) 108.75 ns 91.79 ns 0.84 0.06 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20ShortSingleLine).html>) 99.85 ns 88.37 ns 0.89 0.04 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 91.22248666469108 < 96.9066309165195. IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.362594001839796 (T) = (0 -92.63729163502659) / Math.Sqrt((2.741968738141613 / (299)) + (0.8856802083389247 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10082836067376191 = (103.02514846269062 - 92.63729163502659) / 103.02514846269062 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: LongMultiLine) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.147244224422444 < 17.800800850169615. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 89.83225171111643 (T) = (0 -17070.942372202833) / Math.Sqrt((13220.526489442256 / (299)) + (4270.416014250047 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09222297322730262 = (18805.21523318667 - 17070.942372202833) / 18805.21523318667 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 74.4902115301954 < 87.60747754612393. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.706717548376794 (T) = (0 -76.44451374822526) / Math.Sqrt((3.0297144422812003 / (299)) + (11.467363193334007 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17951565139149575 = (93.16998414152677 - 76.44451374822526) / 93.16998414152677 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 78.54902734807467 < 85.64637150634944. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.31706456138444 (T) = (0 -77.65858961413096) / Math.Sqrt((5.446407326555579 / (299)) + (0.7832398420492211 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1484144786464471 = (91.19294265442241 - 77.65858961413096) / 91.19294265442241 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 74.82643916229443 < 85.73107972889265. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 73.28626330603964 (T) = (0 -75.13919153690213) / Math.Sqrt((2.697865935972899 / (299)) + (0.46886609612119184 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17157283089333425 = (90.70102278022637 - 75.13919153690213) / 90.70102278022637 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: LongMultiLine) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.241009085903084 < 17.960312481197406. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 82.00419317203995 (T) = (0 -17147.026838579655) / Math.Sqrt((12237.94176602182 / (299)) + (6034.575454338784 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09705322449117387 = (18990.074834606956 - 17147.026838579655) / 18990.074834606956 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 89.83377327922298 < 97.71004599069141. IsChangePoint: Marked as a change because one of 2/4/2023 8:08:12 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 74.85040616371522 (T) = (0 -90.67299410830144) / Math.Sqrt((2.9753719593079664 / (299)) + (0.3791795786201535 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14035496333454758 = (105.47724960993243 - 90.67299410830144) / 105.47724960993243 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 77.68899111264507 < 87.26221099707828. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.18389731856834 (T) = (0 -77.96066005769845) / Math.Sqrt((3.043375828417357 / (299)) + (0.8536655498357323 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16313778819106087 = (93.15829889030448 - 77.96066005769845) / 93.15829889030448 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 91.7905080964014 < 97.49197371473718. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.421026900529945 (T) = (0 -92.60887964193708) / Math.Sqrt((3.4978908669599234 / (299)) + (1.114746502507119 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12251647073548796 = (105.53916575455253 - 92.60887964193708) / 105.53916575455253 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 88.37370523800021 < 94.5239924299374. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.910933823613185 (T) = (0 -89.86239328620867) / Math.Sqrt((3.1587829380018193 / (299)) + (1.3503570213038385 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12776515865578816 = (103.0254571666968 - 89.86239328620867) / 103.0254571666968 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Segment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadSingleSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase%3a%20Json40KB).html>) 93.57 μs 80.12 μs 0.86 0.01 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%204096%2c%20TestCase%3a%20Json400KB).html>) 961.29 μs 793.09 μs 0.83 0.01 False
[ReadMultiSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize%3a%204096%2c%20TestCase%3a%20Json400KB).html>) 981.10 μs 868.08 μs 0.88 0.01 False
[ReadMultiSegmentSequenceUsingSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize%3a%208192%2c%20TestCase%3a%20Json400KB).html>) 957.02 μs 792.80 μs 0.83 0.01 False
[ReadMultiSegmentSequenceUsingSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize%3a%208192%2c%20TestCase%3a%20Json4KB).html>) 9.54 μs 7.79 μs 0.82 0.01 False
[ReadMultiSegmentSequenceUsingSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize%3a%204096%2c%20TestCase%3a%20Json4KB).html>) 9.62 μs 7.92 μs 0.82 0.01 False
[ReadSingleSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase%3a%20Json400KB).html>) 943.18 μs 809.95 μs 0.86 0.01 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%204096%2c%20TestCase%3a%20Json40KB).html>) 94.33 μs 78.65 μs 0.83 0.01 False
[ReadMultiSegmentSequenceUsingSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize%3a%204096%2c%20TestCase%3a%20Json400KB).html>) 960.65 μs 798.37 μs 0.83 0.01 False
[ReadSingleSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase%3a%20Json4KB).html>) 9.40 μs 7.87 μs 0.84 0.01 False
[ReadMultiSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize%3a%208192%2c%20TestCase%3a%20Json40KB).html>) 96.99 μs 84.74 μs 0.87 0.01 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%204096%2c%20TestCase%3a%20Json4KB).html>) 9.49 μs 7.70 μs 0.81 0.01 False
[ReadMultiSegmentSequenceUsingSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize%3a%204096%2c%20TestCase%3a%20Json40KB).html>) 95.14 μs 79.87 μs 0.84 0.01 False
[ReadMultiSegmentSequenceUsingSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize%3a%208192%2c%20TestCase%3a%20Json40KB).html>) 95.16 μs 80.35 μs 0.84 0.01 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%208192%2c%20TestCase%3a%20Json400KB).html>) 959.00 μs 789.08 μs 0.82 0.01 False
[ReadMultiSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize%3a%204096%2c%20TestCase%3a%20Json40KB).html>) 97.66 μs 85.09 μs 0.87 0.01 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%208192%2c%20TestCase%3a%20Json4KB).html>) 9.40 μs 7.63 μs 0.81 0.01 False
[ReadMultiSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize%3a%208192%2c%20TestCase%3a%20Json400KB).html>) 979.62 μs 864.22 μs 0.88 0.01 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%208192%2c%20TestCase%3a%20Json40KB).html>) 94.38 μs 78.30 μs 0.83 0.01 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Segment*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 80.11594093406593 < 89.12747441942268. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 159.5344447688933 (T) = (0 -80241.24355202142) / Math.Sqrt((626540.2544959924 / (299)) + (73636.22967077252 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1490364253796637 = (94294.56905699124 - 80241.24355202142) / 94294.56905699124 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json400KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 793.0874603174602 < 916.0955650281442. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 249.29693769955801 (T) = (0 -792769.5493049663) / Math.Sqrt((73318667.81284608 / (299)) + (2807012.6932828934 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1759664449339784 = (962059.7904431812 - 792769.5493049663) / 962059.7904431812 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json400KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 868.0821932392867 < 939.8515807939871. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 128.30908521935015 (T) = (0 -866450.11776566) / Math.Sqrt((55779850.19782754 / (299)) + (9296432.157749968 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12328117003433454 = (988287.3370012966 - 866450.11776566) / 988287.3370012966 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json400KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 792.7990054249548 < 912.5073663372069. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 151.94119020798914 (T) = (0 -796331.8275966246) / Math.Sqrt((85807714.36124752 / (299)) + (12136248.356344268 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17409540910917085 = (964193.4872134479 - 796331.8275966246) / 964193.4872134479 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.7938277472069375 < 9.152986012938122. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 239.19172178579498 (T) = (0 -7745.8696581704535) / Math.Sqrt((6673.001958916331 / (299)) + (511.87353638782014 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19520026079119338 = (9624.59265429853 - 7745.8696581704535) / 9624.59265429853 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.915488055938476 < 9.19289219426407. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 158.20857226602158 (T) = (0 -7811.131201098742) / Math.Sqrt((7101.256815050518 / (299)) + (1525.9222522341843 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19394851778321792 = (9690.610802695592 - 7811.131201098742) / 9690.610802695592 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json400KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 809.9515484515485 < 902.6112850222996. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 109.00029387121047 (T) = (0 -809042.0304231406) / Math.Sqrt((56915482.27233932 / (299)) + (20129612.794029202 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15085547689851678 = (952773.0656121186 - 809042.0304231406) / 952773.0656121186 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 78.65074255204594 < 90.24451265331889. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 129.86667464864644 (T) = (0 -78815.74128549907) / Math.Sqrt((653787.7603529231 / (299)) + (172370.05549237182 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16997380129654885 = (94955.72719103783 - 78815.74128549907) / 94955.72719103783 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json400KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 798.3692144373674 < 916.820407515473. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 218.4069782386053 (T) = (0 -796836.9462571223) / Math.Sqrt((89046582.73046435 / (299)) + (4039072.794625035 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17614910498298073 = (967210.1481915136 - 796836.9462571223) / 967210.1481915136 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.8675173909925435 < 8.941345485600573. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 220.8212091678888 (T) = (0 -7828.911208282243) / Math.Sqrt((6462.04333296129 / (299)) + (421.5626652914961 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1717375916160176 = (9452.211194224283 - 7828.911208282243) / 9452.211194224283 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 84.74175465485922 < 92.25659618882274. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 207.1597490613142 (T) = (0 -84959.87532438866) / Math.Sqrt((571374.8206321459 / (299)) + (19308.260192353206 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1244184923010875 = (97032.51448019838 - 84959.87532438866) / 97032.51448019838 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.6977306365165 < 9.026167092343524. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 295.5010512352461 (T) = (0 -7699.350565702616) / Math.Sqrt((6586.91475464507 / (299)) + (197.14644107624142 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18967355998758414 = (9501.54182996256 - 7699.350565702616) / 9501.54182996256 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 79.87400464650148 < 90.89898942863422. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 253.68454118752942 (T) = (0 -79781.86524202622) / Math.Sqrt((753732.0630332014 / (299)) + (21229.296937046412 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17007968643922233 = (96131.9586210894 - 79781.86524202622) / 96131.9586210894 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 80.35358727810652 < 90.84664537967326. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 160.2604438488747 (T) = (0 -80154.63690398438) / Math.Sqrt((716864.497415057 / (299)) + (93562.87327436416 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16377139631088514 = (95852.54145860755 - 80154.63690398438) / 95852.54145860755 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json400KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 789.0794755877034 < 915.7414674645277. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 245.66475021179383 (T) = (0 -789099.9230631086) / Math.Sqrt((80981576.62707315 / (299)) + (2543177.3737508217 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.175345602613903 = (956885.606339231 - 789099.9230631086) / 956885.606339231 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 85.08853357919254 < 92.50878448470081. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 103.17089629687628 (T) = (0 -85560.21795878319) / Math.Sqrt((424177.5219428213 / (299)) + (146420.97068848825 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11955750936596117 = (97178.65603824749 - 85560.21795878319) / 97178.65603824749 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.6301886527215474 < 9.01680356326642. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 298.798017411369 (T) = (0 -7653.9347746057765) / Math.Sqrt((6095.338131447508 / (299)) + (218.84563767719945 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1923543991229545 = (9476.84821943455 - 7653.9347746057765) / 9476.84821943455 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json400KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 864.2220711814136 < 938.2020567476003. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 156.95030956589784 (T) = (0 -864312.4259645297) / Math.Sqrt((60743143.44878487 / (299)) + (4935293.298857308 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12174938461927988 = (984129.5990323352 - 864312.4259645297) / 984129.5990323352 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json40KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 78.30331696428571 < 90.10602536567218. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 121.38671780300088 (T) = (0 -77933.13658843987) / Math.Sqrt((601060.0810187818 / (299)) + (222021.5509555609 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1770889268823971 = (94704.20211164467 - 77933.13658843987) / 94704.20211164467 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollectionNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollectionNonGeneric(Int32).SortedList(Size%3a%20512).html>) 109.13 μs 84.99 μs 0.78 0.15 False
[Hashtable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollectionNonGeneric(Int32).Hashtable(Size%3a%20512).html>) 34.37 μs 26.00 μs 0.76 0.34 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollectionNonGeneric(Int32).Stack(Size%3a%20512).html>) 26.81 μs 24.04 μs 0.90 0.13 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollectionNonGeneric(Int32).Queue(Size%3a%20512).html>) 27.13 μs 24.70 μs 0.91 0.13 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollectionNonGeneric<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 84.98715304736025 < 102.75421154438233. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 69.74828836302386 (T) = (0 -85919.44301118515) / Math.Sqrt((19668074.536491655 / (299)) + (646043.3249539444 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2162140260318061 = (109621.05200248426 - 85919.44301118515) / 109621.05200248426 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollectionNonGeneric<Int32>.Hashtable(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 26.004599705618844 < 32.653602769161026. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.6715459326085 (T) = (0 -25878.567513855647) / Math.Sqrt((6403079.937998467 / (299)) + (16594.585347228956 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27349213141920736 = (35620.4917152632 - 25878.567513855647) / 35620.4917152632 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollectionNonGeneric<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 24.0418199473549 < 26.86333731009755. IsChangePoint: Marked as a change because one of 5/13/2023 2:42:33 PM, 5/15/2023 9:18:21 AM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.393301781825318 (T) = (0 -24375.43303156937) / Math.Sqrt((478375.16234201705 / (299)) + (301160.6593671642 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.08799049241114731 = (26727.169869108617 - 24375.43303156937) / 26727.169869108617 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollectionNonGeneric<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 24.70279865506329 < 30.102592312628687. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.442159039258774 (T) = (0 -25056.486274367784) / Math.Sqrt((977009.3155104392 / (299)) + (600191.7391777658 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07730897463727798 = (27155.879471698285 - 25056.486274367784) / 27155.879471698285 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array_ComparerClass - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(BigStruct).Array_ComparerClass(Size%3a%20512).html>) 43.05 μs 34.02 μs 0.79 0.07 False
[LinqOrderByExtension - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(BigStruct).LinqOrderByExtension(Size%3a%20512).html>) 54.24 μs 46.51 μs 0.86 0.02 True
[Array_Comparison - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(BigStruct).Array_Comparison(Size%3a%20512).html>) 41.89 μs 35.09 μs 0.84 0.03 False
[LinqQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(BigStruct).LinqQuery(Size%3a%20512).html>) 55.17 μs 45.81 μs 0.83 0.02 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<BigStruct>.Array_ComparerClass(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 34.02201714285714 < 40.71486657142857. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 42.913825770276695 (T) = (0 -33848.1619537898) / Math.Sqrt((674481.522813671 / (299)) + (549380.8542825477 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2110437259433498 = (42902.456152290346 - 33848.1619537898) / 42902.456152290346 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 46.51367714285715 < 51.654520060439545. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.454958053395014 (T) = (0 -45545.413024513946) / Math.Sqrt((387252.8089833673 / (299)) + (2016006.0919830278 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17513914752789236 = (55215.87415382166 - 45545.413024513946) / 55215.87415382166 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 35.086964 < 39.89785896794872. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.782737922071284 (T) = (0 -35245.47168047337) / Math.Sqrt((746695.1010919347 / (299)) + (167583.80622240895 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1665641364651839 = (42289.36289229053 - 35245.47168047337) / 42289.36289229053 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Sort<BigStruct>.LinqQuery(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 45.80854142857144 < 52.386652725641014. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.58601471883734 (T) = (0 -45873.75586249648) / Math.Sqrt((416041.94908381294 / (299)) + (1379112.6816106448 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16963523539592684 = (55245.3064218947 - 45873.75586249648) / 55245.3064218947 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ClassRecord).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 1.14 μs 868.85 ns 0.76 0.16 True
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ClassRecord).DeserializeFromString(Mode%3a%20SourceGen).html>) 1.23 μs 867.43 ns 0.70 0.07 True
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ClassRecord).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 1.08 μs 807.34 ns 0.75 0.12 True
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ClassRecord).DeserializeFromString(Mode%3a%20Reflection).html>) 1.18 μs 886.70 ns 0.75 0.15 True
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ClassRecord).DeserializeFromReader(Mode%3a%20Reflection).html>) 1.64 μs 1.27 μs 0.78 0.19 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ClassRecord).DeserializeFromReader(Mode%3a%20SourceGen).html>) 1.56 μs 1.18 μs 0.76 0.14 True
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ClassRecord).DeserializeFromStream(Mode%3a%20SourceGen).html>) 1.52 μs 1.14 μs 0.75 0.17 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ClassRecord).DeserializeFromStream(Mode%3a%20Reflection).html>) 1.49 μs 1.16 μs 0.78 0.18 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ClassRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 868.8518677513794 < 1.072416644005272. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.88556506161743 (T) = (0 -817.8917797506219) / Math.Sqrt((976.5424869487075 / (299)) + (270.2556055504061 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2510060032398623 = (1091.9870964099975 - 817.8917797506219) / 1091.9870964099975 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 867.4281638547075 < 1.1478564542340275. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.135420015969615 (T) = (0 -892.5352775350152) / Math.Sqrt((939.3797826342582 / (299)) + (559.5638013779009 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23406864899959554 = (1165.2940911339506 - 892.5352775350152) / 1165.2940911339506 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 807.3449431315811 < 1.0284521003494782. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 53.42916614077627 (T) = (0 -822.0958823142198) / Math.Sqrt((783.5970922903464 / (299)) + (277.106824851804 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24125794936624972 = (1083.4985112892482 - 822.0958823142198) / 1083.4985112892482 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 886.6967464931932 < 1.1255471136978716. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 81.13956214561311 (T) = (0 -879.0267589953359) / Math.Sqrt((1247.66465218893 / (299)) + (110.90488122582308 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24755618975534507 = (1168.2291049872852 - 879.0267589953359) / 1168.2291049872852 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2747222789949857 < 1.5464888375232755. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 45.147358126293135 (T) = (0 -1209.7351736944793) / Math.Sqrt((2188.2870352084574 / (299)) + (822.6685524012246 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23872113479022064 = (1589.082830194061 - 1209.7351736944793) / 1589.082830194061 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1807605362528193 < 1.4795299422775952. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 59.6749050518631 (T) = (0 -1194.1792764068925) / Math.Sqrt((1970.8720005847433 / (299)) + (485.4425125313833 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24881041824814956 = (1589.7175698602543 - 1194.1792764068925) / 1589.7175698602543 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.137694234453477 < 1.3958334053238015. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 45.116651751788446 (T) = (0 -1151.340523125869) / Math.Sqrt((1878.273706190283 / (299)) + (608.0068066702057 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22204333666782283 = (1479.9545750972786 - 1151.340523125869) / 1479.9545750972786 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ClassRecord>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1597467780799642 < 1.4195450145874964. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 57.39196896127298 (T) = (0 -1142.5437865326746) / Math.Sqrt((1853.359601726749 / (299)) + (389.55009219075055 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23199558946765708 = (1487.6786784866501 - 1142.5437865326746) / 1487.6786784866501 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Int64.TryParseSpan(value%3a%20%22-9223372036854775808%22).html>) 34.76 ns 32.28 ns 0.93 0.06 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Int64.ToString(value%3a%20-9223372036854775808).html>) 44.15 ns 39.54 ns 0.90 0.01 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Int64.TryParse(value%3a%20%22-9223372036854775808%22).html>) 34.66 ns 32.11 ns 0.93 0.05 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 32.27685351041842 < 32.97957303489084. IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/2/2023 12:51:17 PM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.428334324857982 (T) = (0 -32.15367097463862) / Math.Sqrt((1.0492792310721342 / (299)) + (0.42408556294095484 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12165194469576504 = (36.606981458507924 - 32.15367097463862) / 36.606981458507924 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Int64.ToString(value: -9223372036854775808) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 39.5391518585112 < 42.20633565897377. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.057918317323 (T) = (0 -39.94238632972933) / Math.Sqrt((0.5214125522006336 / (299)) + (0.18507480502511867 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13201853901280006 = (46.01755696982376 - 39.94238632972933) / 46.01755696982376 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 32.108158095285475 < 32.87772707981731. IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/2/2023 4:25:00 AM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.410457048239905 (T) = (0 -32.224801604430134) / Math.Sqrt((1.0414201349132235 / (299)) + (0.05931026237813688 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11874677096120123 = (36.567016769491325 - 32.224801604430134) / 36.567016769491325 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Numerics.Tests.Perf_VectorOf(Int16).GetHashCodeBenchmark.html>) 36.24 ns 32.10 ns 0.89 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 32.100405370743296 < 34.80428385462297. IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 61.1330210832142 (T) = (0 -32.330096346599106) / Math.Sqrt((0.39936256057892633 / (299)) + (0.02031097227037809 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0923891492309115 = (35.62109941634493 - 32.330096346599106) / 35.62109941634493 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 484.84 ns 404.70 ns 0.83 0.02 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromString(Mode%3a%20Reflection).html>) 571.94 ns 467.75 ns 0.82 0.02 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 474.78 ns 375.07 ns 0.79 0.03 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromReader(Mode%3a%20SourceGen).html>) 756.27 ns 608.23 ns 0.80 0.14 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromStream(Mode%3a%20Reflection).html>) 855.10 ns 716.12 ns 0.84 0.07 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromStream(Mode%3a%20SourceGen).html>) 874.99 ns 693.95 ns 0.79 0.08 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromString(Mode%3a%20SourceGen).html>) 569.39 ns 453.39 ns 0.80 0.02 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromReader(Mode%3a%20Reflection).html>) 760.73 ns 664.52 ns 0.87 0.17 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 404.69782584406823 < 463.63168339227155. IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 46.49827043024103 (T) = (0 -397.26135477915767) / Math.Sqrt((106.4274498605255 / (299)) + (40.98826420204147 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1798255753146932 = (484.36203668699267 - 397.26135477915767) / 484.36203668699267 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 467.7494553984167 < 536.0241817122647. IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.80865880707155 (T) = (0 -457.87256078387594) / Math.Sqrt((151.06165481470182 / (299)) + (17.52967562286534 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18395212718712067 = (561.0854167239115 - 457.87256078387594) / 561.0854167239115 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 375.0688455438669 < 459.9222119213518. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.703339917047884 (T) = (0 -387.0624334106987) / Math.Sqrt((142.41737021179833 / (299)) + (65.88782066571808 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19847340232302713 = (482.9065367668443 - 387.0624334106987) / 482.9065367668443 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 608.230622130509 < 724.4701894337308. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.1841996165691 (T) = (0 -624.00411630064) / Math.Sqrt((404.45913652399224 / (299)) + (569.770052961048 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18579607744165152 = (766.3978261611996 - 624.00411630064) / 766.3978261611996 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 716.1205730912292 < 811.1711895146068. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.278160578673464 (T) = (0 -704.8112421934923) / Math.Sqrt((346.40516288851296 / (299)) + (334.7538514387242 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18712774588665762 = (867.0627378742064 - 704.8112421934923) / 867.0627378742064 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 693.9454372174401 < 811.8388204245159. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.39172119550602 (T) = (0 -703.9462522232533) / Math.Sqrt((490.4870013269811 / (299)) + (344.9909196383403 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19630898127012528 = (875.8916496737085 - 703.9462522232533) / 875.8916496737085 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 453.39342096315085 < 538.7978088245292. IsChangePoint: Marked as a change because one of 3/14/2023 10:43:28 AM, 4/22/2023 1:14:29 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 102.82694402526259 (T) = (0 -454.75238376749166) / Math.Sqrt((242.50019503659192 / (299)) + (5.269304284066499 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1996040749300706 = (568.1592940740679 - 454.75238376749166) / 568.1592940740679 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 664.5226752503576 < 717.6507403560084. IsChangePoint: Marked as a change because one of 3/5/2023 5:11:19 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.304393067029586 (T) = (0 -636.1919471959293) / Math.Sqrt((647.6779055720199 / (299)) + (455.28753587227305 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17614020929347415 = (772.2090025177023 - 636.1919471959293) / 772.2090025177023 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Node.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20Json400B).html>) 6.38 μs 5.44 μs 0.85 0.19 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20LotsOfStrings).html>) 8.88 μs 7.03 μs 0.79 0.12 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20LotsOfNumbers).html>) 16.87 μs 13.80 μs 0.82 0.12 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 15.48 μs 13.67 μs 0.88 0.16 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20LotsOfStrings).html>) 10.51 μs 8.19 μs 0.78 0.12 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 771.60 ns 678.56 ns 0.88 0.12 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 800.64 ns 687.38 ns 0.86 0.15 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20DeepTree).html>) 45.76 μs 40.16 μs 0.88 0.14 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20Json4KB).html>) 48.39 μs 40.59 μs 0.84 0.10 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20Json4KB).html>) 43.21 μs 39.19 μs 0.91 0.23 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20Json400B).html>) 6.89 μs 5.35 μs 0.78 0.17 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20BroadTree).html>) 71.79 μs 58.54 μs 0.82 0.13 False

graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Node.Tests.Perf_ParseThenWrite*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.440304341150476 < 6.061458676935356. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.16905313074872 (T) = (0 -5146.187980080498) / Math.Sqrt((43256.52649261163 / (299)) + (85177.52291109494 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20452962469451538 = (6469.364717830261 - 5146.187980080498) / 6469.364717830261 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.0313599909953 < 8.198420130184278. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 25.676768167524987 (T) = (0 -7193.443536393998) / Math.Sqrt((34825.588452885124 / (299)) + (45398.578968791844 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17656564446147138 = (8735.904068138454 - 7193.443536393998) / 8735.904068138454 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.80209873730032 < 16.513767333930065. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.20350183795023 (T) = (0 -13918.145489669969) / Math.Sqrt((140262.4822120939 / (299)) + (87083.99478716422 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1804782521416651 = (16983.253374352065 - 13918.145489669969) / 16983.253374352065 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.674263743455498 < 14.747454078103948. IsChangePoint: Marked as a change because one of 5/19/2023 9:46:56 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.722481656257415 (T) = (0 -13411.36005241701) / Math.Sqrt((166781.5207255237 / (299)) + (543899.2289338904 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13949441653707847 = (15585.442221590065 - 13411.36005241701) / 15585.442221590065 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.185286144578315 < 9.579566342068336. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.243418972088186 (T) = (0 -8175.159425968142) / Math.Sqrt((38223.77199492068 / (299)) + (27319.582826983326 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17702298723807935 = (9933.642494499585 - 8175.159425968142) / 9933.642494499585 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 678.5567452450623 < 737.3823487546834. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.653726354531177 (T) = (0 -667.7811395672533) / Math.Sqrt((446.13632776851495 / (299)) + (309.08980230779343 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14015430888824296 = (776.629046897741 - 667.7811395672533) / 776.629046897741 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 687.38221162763 < 760.9468848654616. IsChangePoint: Marked as a change because one of 4/27/2023 1:05:59 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.318241708956128 (T) = (0 -649.8447926425541) / Math.Sqrt((631.6249542117075 / (299)) + (752.8917161851659 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1280564634703105 = (745.2831122860523 - 649.8447926425541) / 745.2831122860523 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: DeepTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 40.15520052130968 < 43.209620020261674. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.808163820932492 (T) = (0 -38879.433365760204) / Math.Sqrt((836987.3868436916 / (299)) + (994387.3121294458 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14177018422130233 = (45301.890765102034 - 38879.433365760204) / 45301.890765102034 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 40.589384274998665 < 45.927387637070275. IsChangePoint: Marked as a change because one of 5/20/2023 6:43:26 AM, 5/20/2023 9:58:12 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.081747148980348 (T) = (0 -40470.429946866534) / Math.Sqrt((19256160.20630589 / (299)) + (981042.4395344512 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1602955635751281 = (48196.041596699855 - 40470.429946866534) / 48196.041596699855 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 39.19268348320948 < 41.26486606145671. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.147861912301423 (T) = (0 -36502.40298187594) / Math.Sqrt((1385226.0871704903 / (299)) + (6047659.509810116 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16005763374524762 = (43458.22338339444 - 36502.40298187594) / 43458.22338339444 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.347013995344348 < 6.414171070806911. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.74192535065528 (T) = (0 -5478.949806168384) / Math.Sqrt((51208.98477654396 / (299)) + (25294.605064633273 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2104606109011016 = (6939.425545850869 - 5478.949806168384) / 6939.425545850869 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 58.54310767085576 < 68.17719869981983. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 2.559044174248983 (T) = (0 -62849.446926751865) / Math.Sqrt((3042415.5311902002 / (299)) + (156065107.33583683 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12368119154056322 = (71719.8425048537 - 62849.446926751865) / 71719.8425048537 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in GuardedDevirtualization.ThreeClassInterface

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.ThreeClassInterface.Call(testInput%3a%20pB%3d0.00%20pD%3d0.33).html>) 5.83 ns 2.25 ns 0.39 0.14 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.ThreeClassInterface.Call(testInput%3a%20pB%3d0.67%20pD%3d0.00).html>) 4.73 ns 1.93 ns 0.41 0.12 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.ThreeClassInterface.Call(testInput%3a%20pB%3d0.33%20pD%3d0.67).html>) 5.85 ns 2.48 ns 0.42 0.10 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.ThreeClassInterface.Call(testInput%3a%20pB%3d0.00%20pD%3d1.00).html>) 1.91 ns 0.80 ns 0.42 0.12 False
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.ThreeClassInterface.Call(testInput%3a%20pB%3d0.33%20pD%3d0.33).html>) 6.70 ns 4.50 ns 0.67 0.15 False
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.ThreeClassInterface.Call(testInput%3a%20pB%3d0.67%20pD%3d0.33).html>) 4.75 ns 3.73 ns 0.78 0.12 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.ThreeClassInterface.Call(testInput%3a%20pB%3d0.33%20pD%3d0.00).html>) 5.84 ns 2.20 ns 0.38 0.15 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.ThreeClassInterface.Call(testInput%3a%20pB%3d0.00%20pD%3d0.67).html>) 4.73 ns 1.90 ns 0.40 0.13 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.ThreeClassInterface*'
### Payloads [Baseline]() [Compare]() ### Histogram #### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=0.33) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.247498418064795 < 5.542000792190159. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.78970351216651 (T) = (0 -2.4049151508631885) / Math.Sqrt((0.10300958844837854 / (299)) + (0.18972302047500586 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5940176906946867 = (5.923694446140524 - 2.4049151508631885) / 5.923694446140524 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.67 pD=0.00) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.9312972618681061 < 4.499703669028759. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.64256794104476 (T) = (0 -2.0709841596539276) / Math.Sqrt((0.08351133537730764 / (299)) + (0.29106594730385493 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.575431040821425 = (4.877851088456199 - 2.0709841596539276) / 4.877851088456199 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.67) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.480762826641173 < 5.53548323593081. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.906829540212968 (T) = (0 -2.449528124868004) / Math.Sqrt((0.11898570475512604 / (299)) + (0.24790109702077837 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5865535722353066 = (5.924656643211136 - 2.449528124868004) / 5.924656643211136 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=1.00) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0.7989468552082796 < 1.8186990909870018. IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.39319864267503 (T) = (0 -0.7993306128842134) / Math.Sqrt((0.10178804986925455 / (299)) + (2.474002260442509E-07 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5978136465621858 = (1.9874632892232265 - 0.7993306128842134) / 1.9874632892232265 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.33) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.5029947454114865 < 6.3739081218223035. IsChangePoint: Marked as a change because one of 3/29/2023 12:02:44 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.866626931892903 (T) = (0 -4.422997241456417) / Math.Sqrt((0.16483056280777475 / (299)) + (0.26904629453076584 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35726674723407187 = (6.881544128022878 - 4.422997241456417) / 6.881544128022878 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.67 pD=0.33) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.725583408351222 < 4.506327594201768. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.806477024993463 (T) = (0 -2.4118803282018595) / Math.Sqrt((0.055468530838924404 / (299)) + (0.4061719398788948 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5036817189037923 = (4.859543603501347 - 2.4118803282018595) / 4.859543603501347 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.00) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.199478797717434 < 5.538917919654002. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.06793164140992 (T) = (0 -2.633234841940612) / Math.Sqrt((0.07368389861683776 / (299)) + (0.42820474687516247 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5555450105793747 = (5.924637825245695 - 2.633234841940612) / 5.924637825245695 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=0.67) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.8969632785432278 < 4.4968177999933925. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.46454728755007 (T) = (0 -2.538870106672351) / Math.Sqrt((0.08642688701038223 / (299)) + (0.7901155413112382 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.47949735002979477 = (4.877727532833275 - 2.538870106672351) / 4.877727532833275 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22http%3a%2f%2fhost%2fpath%20with%20escapable%20values%3fkey%3dva%20lue%22).html>) 802.26 ns 637.27 ns 0.79 0.01 True
[UnescapeDataString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.UnescapeDataString(input%3a%20%22%25E4%25BD%25A0%25E5%25A5%25BD%22).html>) 76.04 ns 63.67 ns 0.84 0.00 True
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22https%3a%2f%2fcontoso.com%2fpath%2fwith%3fkey%3dvalue%23fragment%22).html>) 589.20 ns 465.80 ns 0.79 0.02 False
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22http%3a%2f%2fhost%2f%22).html>) 335.90 ns 239.14 ns 0.71 0.01 True
[UnescapeDataString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.UnescapeDataString(input%3a%20%22abc%2520def%2520ghi%2520%22).html>) 79.03 ns 53.71 ns 0.68 0.00 True
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.Ctor(input%3a%20%22http%3a%2f%2fxn--hst-sna.with.xn--nicode-2ya%22).html>) 221.89 ns 191.33 ns 0.86 0.01 False
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22http%3a%2f%2fhost%2fpath%3fkey1%3dvalue1%26key2%3dvalue2%26key3%3dvalue3%26key4%3dvalue4%22).html>) 667.99 ns 560.48 ns 0.84 0.01 False
[BuilderToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.BuilderToString.html>) 198.88 ns 156.12 ns 0.78 0.04 True
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.Ctor(input%3a%20%22https%3a%2f%2fa.much.longer.domain.name%22).html>) 234.86 ns 203.87 ns 0.87 0.02 False
[CombineAbsoluteRelative - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CombineAbsoluteRelative(input%3a%20%22%2fnew%2fpath%22).html>) 313.29 ns 219.64 ns 0.70 0.01 True
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22https%3a%2f%2fCONTOSO.com%2fpath%2fwith%3fkey%3dvalue%23fragment%22).html>) 610.58 ns 468.92 ns 0.77 0.01 False
[DnsSafeHost - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.DnsSafeHost.html>) 346.21 ns 290.18 ns 0.84 0.02 True
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.Ctor(input%3a%20%22https%3a%2f%2fcontoso.com%22).html>) 155.51 ns 126.25 ns 0.81 0.01 True
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22http%3a%2f%2fxn--hst-sna.with.xn--nicode-2ya%2fpath%2fwith%3fkey%3dvalue%23fragment%22).html>) 696.38 ns 572.32 ns 0.82 0.01 False
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.Ctor(input%3a%20%22http%3a%2f%2fh%c3%b6st.with.%c3%bcnicode%22).html>) 578.32 ns 487.58 ns 0.84 0.07 False
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22http%3a%2f%2fhost%2fpath%2520with%2520escapable%2520values%3fkey%3dva%2520lue%22).html>) 652.13 ns 503.40 ns 0.77 0.00 False
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22http%3a%2f%2fh%c3%b6st.with.%c3%bcnicode%2fpath%2fwith%3fkey%3dvalue%23fragment%22).html>) 2.90 μs 2.72 μs 0.94 0.01 False
[ParseAbsoluteUri - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.ParseAbsoluteUri.html>) 446.86 ns 377.03 ns 0.84 0.00 True
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22https%3a%2f%2fa.much.longer.domain.name%2fpath%2fwith%3fkey%3dvalue%23fragment%22).html>) 697.58 ns 585.64 ns 0.84 0.01 False
[UriBuilderReplacePort - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.UriBuilderReplacePort.html>) 224.48 ns 140.30 ns 0.62 0.01 True
[GetComponents - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.GetComponents.html>) 29.20 ns 21.43 ns 0.73 0.23 False
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22http%3a%2f%2fhost%2fpath%20with%20escapable%20values%3fkey%3d%c3%bcnicode%22).html>) 1.12 μs 933.71 ns 0.83 0.01 False
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22http%3a%2f%2fhost%2fpath%2520with%2520escapable%2520values%3fkey%3d%25C3%25BCnicode%22).html>) 1.20 μs 1.01 μs 0.84 0.01 False
[CtorIdnHostPathAndQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input%3a%20%22http%3a%2f%2fdot.net%2fpath%2fwith%3fkey%3dvalue%23fragment%22).html>) 576.36 ns 455.69 ns 0.79 0.01 False
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.Ctor(input%3a%20%22http%3a%2f%2fdot.net%22).html>) 146.91 ns 115.73 ns 0.79 0.03 True
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Uri.Ctor(input%3a%20%22https%3a%2f%2fCONTOSO.com%22).html>) 155.66 ns 125.70 ns 0.81 0.01 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path with escapable values?key=va lue") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 637.2696769098671 < 762.4583341011509. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.486157527471725 (T) = (0 -630.5075124401646) / Math.Sqrt((365.0030553676675 / (299)) + (137.25127216352172 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23516189206107949 = (824.3672822988007 - 630.5075124401646) / 824.3672822988007 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.UnescapeDataString(input: "%E4%BD%A0%E5%A5%BD") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 63.66676570405445 < 72.57656130718823. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 76.52550756737772 (T) = (0 -63.904452371965625) / Math.Sqrt((0.607268865053893 / (299)) + (0.3858702136638101 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17576988203537694 = (77.53229465792037 - 63.904452371965625) / 77.53229465792037 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "https://contoso.com/path/with?key=value#fragment") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 465.80142026531786 < 562.1377660954812. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 77.28485915362016 (T) = (0 -470.0248114146007) / Math.Sqrt((573.8167305662658 / (299)) + (17.799874272835414 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22968408505512666 = (610.1714923652297 - 470.0248114146007) / 610.1714923652297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 239.13856708179543 < 321.35780182299254. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 211.25840031327382 (T) = (0 -238.39251416282798) / Math.Sqrt((60.889477700698414 / (299)) + (0.588340790497858 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3065736340573773 = (343.78922676060125 - 238.39251416282798) / 343.78922676060125 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.UnescapeDataString(input: "abc%20def%20ghi%20") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 53.71063509971177 < 75.27181957913461. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 144.48167475777313 (T) = (0 -53.4003485835231) / Math.Sqrt((1.1031414747677795 / (299)) + (0.3976573173817455 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33374758282953454 = (80.15032622367242 - 53.4003485835231) / 80.15032622367242 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.Ctor(input: "http://xn--hst-sna.with.xn--nicode-2ya") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 191.33282154495095 < 211.50860472397162. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 59.113534050988754 (T) = (0 -189.6802476384151) / Math.Sqrt((141.9871711146311 / (299)) + (1.3133589477045724 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.191268384163482 = (234.54041356132444 - 189.6802476384151) / 234.54041356132444 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path?key1=value1&key2=value2&key3=value3&key4=value4") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 560.4808363935906 < 639.4603169035769. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.2023308749477 (T) = (0 -568.2693405601) / Math.Sqrt((3947.603085418307 / (299)) + (67.53544625443547 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2146098935795478 = (723.5504189759702 - 568.2693405601) / 723.5504189759702 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.BuilderToString ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 156.11680194694142 < 189.43004710559953. IsChangePoint: Marked as a change because one of 4/26/2023 8:38:54 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 95.24052837966903 (T) = (0 -155.3396420083473) / Math.Sqrt((19.21127395779752 / (299)) + (1.2792556813738118 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19825013185239768 = (193.75075466772543 - 155.3396420083473) / 193.75075466772543 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.Ctor(input: "https://a.much.longer.domain.name") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 203.86502087044317 < 223.10748001776972. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 42.76697831769251 (T) = (0 -205.01545148210073) / Math.Sqrt((122.3015151962871 / (299)) + (5.7893630615953855 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16164838601032153 = (244.54590181611414 - 205.01545148210073) / 244.54590181611414 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CombineAbsoluteRelative(input: "/new/path") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 219.63911597328536 < 297.2501930952115. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 165.61115243354095 (T) = (0 -221.20118156930786) / Math.Sqrt((50.66433938945897 / (299)) + (2.225036629216261 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.30407882423555055 = (317.8537875734629 - 221.20118156930786) / 317.8537875734629 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "https://CONTOSO.com/path/with?key=value#fragment") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 468.92275554262073 < 577.1568906920705. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 85.57997418879727 (T) = (0 -474.0824024048639) / Math.Sqrt((579.7062095115274 / (299)) + (14.017624768232778 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23870680439113787 = (622.7330089633933 - 474.0824024048639) / 622.7330089633933 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.DnsSafeHost ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 290.1784769848043 < 329.66145302213687. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 67.71376209855275 (T) = (0 -294.7486728088325) / Math.Sqrt((44.88745075630595 / (299)) + (8.800582271547283 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17282273661769337 = (356.33072360283785 - 294.7486728088325) / 356.33072360283785 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.Ctor(input: "https://contoso.com") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 126.25223688311021 < 147.58284165249069. IsChangePoint: Marked as a change because one of 3/21/2023 8:47:40 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.401659085545994 (T) = (0 -126.57361386071683) / Math.Sqrt((10.665002246229271 / (299)) + (8.392726378397033 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20441598566213243 = (159.09522008943196 - 126.57361386071683) / 159.09522008943196 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://xn--hst-sna.with.xn--nicode-2ya/path/with?key=value#fragment") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 572.3183334942582 < 662.527266819423. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 74.19707398413365 (T) = (0 -574.2973074374005) / Math.Sqrt((950.3267102937127 / (299)) + (11.933491177463985 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.207283359124568 = (724.4673289602936 - 574.2973074374005) / 724.4673289602936 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.Ctor(input: "http://höst.with.ünicode") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 487.57988111140975 < 549.2795203576752. IsChangePoint: Marked as a change because one of 2/19/2023 7:20:18 PM, 3/19/2023 10:05:03 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.31237366586805 (T) = (0 -486.05017664733157) / Math.Sqrt((336.9725675345963 / (299)) + (86.41866050994823 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1723985022083559 = (587.299778872197 - 486.05017664733157) / 587.299778872197 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path%20with%20escapable%20values?key=va%20lue") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 503.4027150653079 < 619.402924014478. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 96.18474230854856 (T) = (0 -496.85737826673534) / Math.Sqrt((507.69461843330004 / (299)) + (21.279524105963542 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2611853707553901 = (672.506145113477 - 496.85737826673534) / 672.506145113477 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://höst.with.ünicode/path/with?key=value#fragment") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.7176060534581437 < 2.7433481659616454. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.71783789101441 (T) = (0 -2647.681981270366) / Math.Sqrt((2756.0422577253494 / (299)) + (1097.5224036161874 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10388173553870624 = (2954.6122272845696 - 2647.681981270366) / 2954.6122272845696 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.ParseAbsoluteUri ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 377.0331626000359 < 420.2113417806379. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 103.4340706365068 (T) = (0 -380.9469093973986) / Math.Sqrt((65.50805073584887 / (299)) + (3.403225849910966 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15845119110344938 = (452.6735768266382 - 380.9469093973986) / 452.6735768266382 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "https://a.much.longer.domain.name/path/with?key=value#fragment") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 585.6420711380734 < 662.8984347627405. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 77.06715517238365 (T) = (0 -580.6882495879662) / Math.Sqrt((859.9140766396744 / (299)) + (6.729666071232103 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19645713411024113 = (722.6599528638377 - 580.6882495879662) / 722.6599528638377 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.UriBuilderReplacePort ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 140.29759560366068 < 213.17330453630436. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 343.31413253075436 (T) = (0 -140.4332955565191) / Math.Sqrt((14.242700537540887 / (299)) + (0.09689636848052004 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36458970955247366 = (221.01199440382123 - 140.4332955565191) / 221.01199440382123 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.GetComponents ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 21.42690516043329 < 27.819301610529863. IsChangePoint: Marked as a change because one of 5/1/2023 10:26:58 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.055179329360666 (T) = (0 -24.907191580202365) / Math.Sqrt((4.088829807136206 / (299)) + (2.066405316457041 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24119811046120293 = (32.824366838808295 - 24.907191580202365) / 32.824366838808295 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path with escapable values?key=ünicode") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 933.7146658366615 < 1.0672384010739615. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 69.64744672877235 (T) = (0 -939.8809522327228) / Math.Sqrt((1208.6194534687734 / (299)) + (72.39271722803913 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18681201112824783 = (1155.7978783439107 - 939.8809522327228) / 1155.7978783439107 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://host/path%20with%20escapable%20values?key=%C3%BCnicode") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.01198199591535 < 1.14427806231839. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.50516851109161 (T) = (0 -1009.6449304234995) / Math.Sqrt((1210.9518722358573 / (299)) + (72.97814299009873 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18862581622410013 = (1244.3641301537414 - 1009.6449304234995) / 1244.3641301537414 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.CtorIdnHostPathAndQuery(input: "http://dot.net/path/with?key=value#fragment") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 455.69081576019283 < 547.0505038091819. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 79.96231280917591 (T) = (0 -445.8447204498892) / Math.Sqrt((547.9426897430368 / (299)) + (20.70608592924145 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24921338181214897 = (593.8367968331801 - 445.8447204498892) / 593.8367968331801 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.Ctor(input: "http://dot.net") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 115.7342372868377 < 140.81090339324285. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 108.59037795102974 (T) = (0 -114.71091224054148) / Math.Sqrt((7.995321038688531 / (299)) + (1.1841511701482579 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24525271181680067 = (151.9858554466217 - 114.71091224054148) / 151.9858554466217 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Uri.Ctor(input: "https://CONTOSO.com") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 125.70285913553553 < 148.25884766832402. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 67.75678364174462 (T) = (0 -125.44750956324586) / Math.Sqrt((10.431503804152298 / (299)) + (2.746799827765543 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2113497747029727 = (159.06609234277323 - 125.44750956324586) / 159.06609234277323 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Where01LinqQueryX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Where01LinqQueryX.html>) 354.50 ms 332.88 ms 0.94 0.00 False
[Where01LinqMethodNestedX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Where01LinqMethodNestedX.html>) 418.61 ms 356.50 ms 0.85 0.25 False
[Order00LinqMethodX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Order00LinqMethodX.html>) 118.24 ms 90.19 ms 0.76 0.02 False
[Where00ForX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Where00ForX.html>) 336.87 ms 276.35 ms 0.82 0.00 True
[Where01LinqMethodX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Where01LinqMethodX.html>) 351.65 ms 324.23 ms 0.92 0.00 False
[Where00LinqQueryX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Where00LinqQueryX.html>) 734.86 ms 590.91 ms 0.80 0.23 False
[Count00ForX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Count00ForX.html>) 229.10 ms 189.55 ms 0.83 0.00 True
[Where00LinqMethodX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Where00LinqMethodX.html>) 729.33 ms 592.93 ms 0.81 0.22 False
[Count00LinqMethodX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Count00LinqMethodX.html>) 1.04 secs 863.46 ms 0.83 0.34 False
[Order00ManualX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Order00ManualX.html>) 150.64 ms 135.18 ms 0.90 0.19 False
[Order00LinqQueryX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/LinqBenchmarks.Order00LinqQueryX.html>) 117.21 ms 89.65 ms 0.76 0.01 False

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'
### Payloads [Baseline]() [Compare]() ### Histogram #### LinqBenchmarks.Where01LinqQueryX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 332.88474285714284 < 337.00328499999995. IsChangePoint: Marked as a change because one of 3/31/2023 8:29:50 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 51.652229521383624 (T) = (0 -328522230.42828965) / Math.Sqrt((20602404068857.754 / (299)) + (2404675564390.992 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07340536876936936 = (354547953.717336 - 328522230.42828965) / 354547953.717336 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### LinqBenchmarks.Where01LinqMethodNestedX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 356.5039733333333 < 402.0072725. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 33.50149662543651 (T) = (0 -356280665.49591434) / Math.Sqrt((1705245692760178 / (299)) + (7324000980817.847 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19053835048152398 = (440145207.2594851 - 356280665.49591434) / 440145207.2594851 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### LinqBenchmarks.Order00LinqMethodX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 90.1908525 < 112.18853321428571. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.886971936503302 (T) = (0 -91400174.99753451) / Math.Sqrt((60742883373630.43 / (299)) + (21974378121781.99 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15179473516747022 = (107757141.79937403 - 91400174.99753451) / 107757141.79937403 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### LinqBenchmarks.Where00ForX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 276.3460615384616 < 317.76759949999996. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 79.22081252938744 (T) = (0 -278330803.58692586) / Math.Sqrt((22955849649168.387 / (299)) + (6681816326702.297 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17949937799183527 = (339220710.0412853 - 278330803.58692586) / 339220710.0412853 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### LinqBenchmarks.Where01LinqMethodX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 324.22983571428574 < 337.7158311428571. IsChangePoint: Marked as a change because one of 3/31/2023 7:02:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.19070369929712 (T) = (0 -327723722.3992674) / Math.Sqrt((18363501315032.664 / (299)) + (3133748408139.9453 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07482126216498053 = (354227468.7009809 - 327723722.3992674) / 354227468.7009809 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### LinqBenchmarks.Where00LinqQueryX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 590.9140666666666 < 697.4294216666666. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 83.34821154654563 (T) = (0 -591380402.6284953) / Math.Sqrt((763263198987104.6 / (299)) + (10810584993428.6 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20589353876246788 = (744711737.6515117 - 591380402.6284953) / 744711737.6515117 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### LinqBenchmarks.Count00ForX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 189.55085384615384 < 216.96982717032967. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 108.84317503084213 (T) = (0 -190315616.73851788) / Math.Sqrt((7180727684216.892 / (299)) + (1418977337262.7644 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1726662035205587 = (230034863.25394794 - 190315616.73851788) / 230034863.25394794 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### LinqBenchmarks.Where00LinqMethodX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 592.9332733333333 < 701.8460205238094. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.835676471509714 (T) = (0 -592888139.7435896) / Math.Sqrt((632275559601832.2 / (299)) + (170761199493399.34 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20369264185604732 = (744546855.7838067 - 592888139.7435896) / 744546855.7838067 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### LinqBenchmarks.Count00LinqMethodX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 863.46258 < 1.0096449410384614. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.559768070119087 (T) = (0 -794771430.3747535) / Math.Sqrt((42239098213501200 / (299)) + (900036227346201.6 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28242198469122387 = (1107574944.3532782 - 794771430.3747535) / 1107574944.3532782 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### LinqBenchmarks.Order00ManualX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 135.17994 < 143.49152348214284. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.128270475520374 (T) = (0 -135232938.7404903) / Math.Sqrt((20026185324667.08 / (299)) + (222556635294.65747 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09706180858512116 = (149769873.5376162 - 135232938.7404903) / 149769873.5376162 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### LinqBenchmarks.Order00LinqQueryX ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 89.65425125 < 111.13591625. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 27.701535600597513 (T) = (0 -91288573.36538461) / Math.Sqrt((61190920743264.11 / (299)) + (1486796475460.5168 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14631665940621694 = (106934935.97038974 - 91288573.36538461) / 106934935.97038974 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tasks.ValueTaskPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateAndAwait_FromCompletedValueTaskSource - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedValueTaskSource.html>) 25.25 ns 19.81 ns 0.78 0.02 False
[Copy_PassAsArgumentAndReturn_FromValueTaskSource - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Threading.Tasks.ValueTaskPerfTest.Copy_PassAsArgumentAndReturn_FromValueTaskSource.html>) 10.95 ns 8.40 ns 0.77 0.08 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedValueTaskSource ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.810830713342874 < 23.25546570174072. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 123.3770760661592 (T) = (0 -19.657998362570474) / Math.Sqrt((1.297171135284713 / (299)) + (0.02037812580781023 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3253835705097767 = (29.13951914486447 - 19.657998362570474) / 29.13951914486447 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Threading.Tasks.ValueTaskPerfTest.Copy_PassAsArgumentAndReturn_FromValueTaskSource ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.400448008625444 < 10.49785418585467. IsChangePoint: Marked as a change because one of 4/27/2023 1:05:59 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 7.9784990514120935 (T) = (0 -9.429060090514058) / Math.Sqrt((0.17718945152269708 / (299)) + (0.17555065255455968 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09129202320386885 = (10.376336877506546 - 9.429060090514058) / 10.376336877506546 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForEachNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ArrayList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEachNonGeneric(String).ArrayList(Size%3a%20512).html>) 4.13 μs 2.33 μs 0.56 0.12 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEachNonGeneric(String).Stack(Size%3a%20512).html>) 4.46 μs 2.34 μs 0.53 0.11 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEachNonGeneric(String).Queue(Size%3a%20512).html>) 5.16 μs 2.64 μs 0.51 0.13 False
[Hashtable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEachNonGeneric(String).Hashtable(Size%3a%20512).html>) 12.79 μs 11.34 μs 0.89 0.19 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEachNonGeneric<String>.ArrayList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.325827539429269 < 3.9250696113392287. IsChangePoint: Marked as a change because one of 2/6/2023 10:25:44 PM, 2/8/2023 10:48:55 AM, 2/14/2023 6:31:57 PM, 2/15/2023 12:21:17 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.87956158648141 (T) = (0 -2329.1902563612102) / Math.Sqrt((1284288.1211106463 / (299)) + (50.394557110541456 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.47297036989228847 = (4419.4673758384 - 2329.1902563612102) / 4419.4673758384 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEachNonGeneric<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.34144485286209 < 3.9627197284884312. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 3/18/2023 3:39:51 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.757575912305597 (T) = (0 -2339.136508409722) / Math.Sqrt((1330940.958787801 / (299)) + (12.470606728043663 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4753147119200632 = (4458.1705672931885 - 2339.136508409722) / 4458.1705672931885 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEachNonGeneric<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.6392530061293806 < 4.911515390335817. IsChangePoint: Marked as a change because one of 4/7/2023 6:04:01 AM, 4/8/2023 9:16:19 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.150942008808386 (T) = (0 -2643.909589810148) / Math.Sqrt((1032575.1596680258 / (299)) + (20.027963079712134 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5170211984156503 = (5474.173154467947 - 2643.909589810148) / 5474.173154467947 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEachNonGeneric<String>.Hashtable(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.34089987458194 < 12.401519210538517. IsChangePoint: Marked as a change because one of 1/24/2023 3:05:55 AM, 1/24/2023 11:24:10 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.266412513526745 (T) = (0 -11522.609500432272) / Math.Sqrt((1588795.921903197 / (299)) + (69695.33782394054 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1537772837617285 = (13616.521134830708 - 11522.609500432272) / 13616.521134830708 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchF.Bisect

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Benchstone.BenchF.Bisect.Test.html>) 319.96 ms 286.00 ms 0.89 0.03 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Bisect*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.Bisect.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 286.0039583333333 < 303.1269941071429. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 252.19242089143424 (T) = (0 -286394635.9397014) / Math.Sqrt((3510293665450.3496 / (299)) + (66592288054.65804 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10261443362591893 = (319143350.0506246 - 286394635.9397014) / 319143350.0506246 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.Trim(s%3a%20%22%20Test%22).html>) 15.26 ns 13.65 ns 0.89 0.34 False
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.Trim(s%3a%20%22%20Te%20st%20%20%22).html>) 18.43 ns 15.93 ns 0.86 0.25 False
[Format_OneArg - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.Format_OneArg(s%3a%20%22Testing%20%7b0%7d%2c%20%7b0%3aC%7d%2c%20%7b0%3aD5%7d%2c%20%7b0%3aE%7d%20-%20%7b0%3aF4%7d%7b0%3aG%7d%7b0%3aN%7d%20%20%7b0%3aX%7d%20!!%22%2c%20o%3a%208).html>) 535.60 ns 497.12 ns 0.93 0.01 False
[ToLowerInvariant - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.ToLowerInvariant(s%3a%20%22test%22).html>) 9.95 ns 5.73 ns 0.58 0.04 True
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.Trim(s%3a%20%22Test%20%22).html>) 17.63 ns 15.40 ns 0.87 0.30 False
[Concat_CharEnumerable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.Concat_CharEnumerable.html>) 7.97 μs 4.90 μs 0.62 0.09 False
[ToUpperInvariant - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.ToUpperInvariant(s%3a%20%22TEST%22).html>) 9.27 ns 4.84 ns 0.52 0.06 True
[Join_Enumerable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.Join_Enumerable.html>) 331.80 ns 280.12 ns 0.84 0.08 False
[Format_OneArg - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.Format_OneArg(s%3a%20%22Testing%20%7b0%7d%2c%20%7b0%3aC%7d%2c%20%7b0%3aE%7d%20-%20%7b0%3aF4%7d%7b0%3aG%7d%7b0%3aN%7d%20%2c%20!!%22%2c%20o%3a%203.14159).html>) 1.11 μs 916.63 ns 0.82 0.00 False
[TrimStart - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.TrimStart(s%3a%20%22Test%22).html>) 3.61 ns 2.26 ns 0.63 0.59 False
[CtorCharCount - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_String.CtorCharCount(size%3a%201).html>) 9.39 ns 7.59 ns 0.81 0.07 False

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Trim(s: " Test") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.652079803512144 < 15.522016188017526. IsChangePoint: Marked as a change because one of 4/4/2023 3:37:20 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 30.459088691232733 (T) = (0 -13.485900640211044) / Math.Sqrt((1.4617536257841275 / (299)) + (0.12767773377504107 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21502993518156 = (17.180146408933787 - 13.485900640211044) / 17.180146408933787 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Trim(s: " Te st ") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.934703797374391 < 17.83880224516524. IsChangePoint: Marked as a change because one of 4/4/2023 3:37:20 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.3118435520911 (T) = (0 -15.85060212632509) / Math.Sqrt((1.147407439807172 / (299)) + (0.0638308341506162 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18828804169574487 = (19.52737293588545 - 15.85060212632509) / 19.52737293588545 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:D5}, {0:E} - {0:F4}{0:G}{0:N} {0:X} !!", o: 8) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 497.11612253381514 < 508.526792622829. IsChangePoint: Marked as a change because one of 3/2/2023 11:15:04 AM, 4/25/2023 6:46:56 PM, 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 43.18121571338835 (T) = (0 -497.2697056865181) / Math.Sqrt((199.30099333057467 / (299)) + (4.131929369340275 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07932243799915588 = (540.1127671731639 - 497.2697056865181) / 540.1127671731639 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.ToLowerInvariant(s: "test") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.7346103687463374 < 8.545183531468268. IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 51.92896456530255 (T) = (0 -5.728543713297323) / Math.Sqrt((0.12226291581172032 / (299)) + (0.05525503855110234 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3822449613930424 = (9.27316388420762 - 5.728543713297323) / 9.27316388420762 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Trim(s: "Test ") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.399152148747625 < 16.718665608173943. IsChangePoint: Marked as a change because one of 4/4/2023 3:37:20 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.66639958683857 (T) = (0 -14.46151693036755) / Math.Sqrt((2.1333823837728705 / (299)) + (0.19774209933205925 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1829806029647014 = (17.700334879249816 - 14.46151693036755) / 17.700334879249816 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Concat_CharEnumerable ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.904641791547021 < 7.598191180055903. IsChangePoint: Marked as a change because one of 5/22/2023 10:35:01 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 165.45109063179558 (T) = (0 -4903.420305222329) / Math.Sqrt((108791.0572793015 / (299)) + (2.98405768281592 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.3920755484310182 = (8065.838267513629 - 4903.420305222329) / 8065.838267513629 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.ToUpperInvariant(s: "TEST") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.844113644550876 < 8.801971582716051. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 30.187219284395276 (T) = (0 -5.576700953871797) / Math.Sqrt((0.07158007243535204 / (299)) + (0.18887551156369445 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39680024050692975 = (9.245197575275002 - 5.576700953871797) / 9.245197575275002 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Join_Enumerable ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 280.12336139342915 < 315.5037956475752. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.280119015811295 (T) = (0 -281.6911773753374) / Math.Sqrt((45.110695983428386 / (299)) + (17.440045241441204 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1518411626299402 = (332.120783235361 - 281.6911773753374) / 332.120783235361 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:E} - {0:F4}{0:G}{0:N} , !!", o: 3.14159) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 916.6338594850786 < 1.0574754354329097. IsChangePoint: Marked as a change because one of 3/2/2023 11:15:04 AM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 61.767819859876546 (T) = (0 -918.0378020418004) / Math.Sqrt((1315.6769757640523 / (299)) + (43.15936284354196 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15750170265595695 = (1089.6613143740396 - 918.0378020418004) / 1089.6613143740396 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.TrimStart(s: "Test") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.260831859731033 < 3.389752310174241. IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.97494709312975 (T) = (0 -2.392618691237963) / Math.Sqrt((0.11391360217418772 / (299)) + (0.03187073351271929 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21034353063312689 = (3.029948824653213 - 2.392618691237963) / 3.029948824653213 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.CtorCharCount(size: 1) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.592075266675955 < 8.915174954420667. IsChangePoint: Marked as a change because one of 3/24/2023 7:13:05 PM, 3/25/2023 2:21:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.187835411733115 (T) = (0 -7.700759272367626) / Math.Sqrt((0.5407481802771312 / (299)) + (0.079100925503654 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18890897891261427 = (9.494321934477385 - 7.700759272367626) / 9.494321934477385 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromString(Mode%3a%20SourceGen).html>) 13.93 μs 11.63 μs 0.83 0.00 True
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromStream(Mode%3a%20SourceGen).html>) 15.53 μs 12.73 μs 0.82 0.05 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromStream(Mode%3a%20Reflection).html>) 15.51 μs 13.04 μs 0.84 0.08 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromString(Mode%3a%20Reflection).html>) 13.92 μs 11.70 μs 0.84 0.00 True
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromReader(Mode%3a%20SourceGen).html>) 17.49 μs 14.11 μs 0.81 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 13.71 μs 11.29 μs 0.82 0.01 True
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromReader(Mode%3a%20Reflection).html>) 17.64 μs 14.04 μs 0.80 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 13.67 μs 11.16 μs 0.82 0.00 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;HashSet&lt;String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.629983411758484 < 13.163232849853504. IsChangePoint: Marked as a change because one of 3/15/2023 4:14:49 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 87.12327450127096 (T) = (0 -11634.549456192071) / Math.Sqrt((36013.173511856636 / (299)) + (9405.050916899527 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17866964666151952 = (14165.49310384165 - 11634.549456192071) / 14165.49310384165 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.725177888533633 < 14.761403053839006. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 78.48670451513382 (T) = (0 -12818.16573202585) / Math.Sqrt((46960.02364995756 / (299)) + (16541.89754212485 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18798667161932903 = (15785.659279248563 - 12818.16573202585) / 15785.659279248563 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.03652206685034 < 14.718582820543512. IsChangePoint: Marked as a change because one of 2/9/2023 1:20:08 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 68.07378327337776 (T) = (0 -12877.190318398572) / Math.Sqrt((73289.46878197962 / (299)) + (20623.75355266613 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1844988531614273 = (15790.52386170046 - 12877.190318398572) / 15790.52386170046 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.702261178861791 < 13.256680992127551. IsChangePoint: Marked as a change because one of 3/15/2023 2:24:48 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 124.04664366481339 (T) = (0 -11630.331684735846) / Math.Sqrt((39612.33990808404 / (299)) + (3751.4460621342637 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.179559941455717 = (14175.723824811392 - 11630.331684735846) / 14175.723824811392 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.105089979091993 < 16.494712576512054. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 105.80017586338276 (T) = (0 -14094.013944735554) / Math.Sqrt((36158.89096208254 / (299)) + (12618.87200191131 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19873070824278347 = (17589.60949798388 - 14094.013944735554) / 17589.60949798388 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.286304826808772 < 12.862960952019804. IsChangePoint: Marked as a change because one of 3/14/2023 10:43:28 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 112.67519932197672 (T) = (0 -11336.708451190043) / Math.Sqrt((50870.15609164116 / (299)) + (4011.390206400832 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17861607120205159 = (13801.960391142193 - 11336.708451190043) / 13801.960391142193 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.038544331116505 < 16.570403661456865. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 125.56222652624807 (T) = (0 -14130.106909147637) / Math.Sqrt((29482.92364786398 / (299)) + (8453.03350302976 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1956039263470036 = (17566.106265261482 - 14130.106909147637) / 17566.106265261482 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.164861509073544 < 12.750611637427792. IsChangePoint: Marked as a change because one of 3/13/2023 9:17:14 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 116.75291908303326 (T) = (0 -11243.830039646098) / Math.Sqrt((35785.278360382275 / (299)) + (4461.457797045742 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1826061573439031 = (13755.706799930873 - 11243.830039646098) / 13755.706799930873 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToLower_Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Perf_Ascii.ToLower_Bytes(Size%3a%206).html>) 10.86 ns 9.66 ns 0.89 0.19 True
[ToUpper_Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Perf_Ascii.ToUpper_Bytes(Size%3a%20128).html>) 19.14 ns 17.63 ns 0.92 0.05 True
[ToUpper_Bytes_Chars - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Perf_Ascii.ToUpper_Bytes_Chars(Size%3a%206).html>) 11.77 ns 9.67 ns 0.82 0.08 True
[ToLower_Chars - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Perf_Ascii.ToLower_Chars(Size%3a%206).html>) 11.43 ns 9.74 ns 0.85 0.24 True
[ToLower_Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Perf_Ascii.ToLower_Bytes(Size%3a%20128).html>) 19.34 ns 17.69 ns 0.91 0.06 True
[ToUpper_Chars - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Perf_Ascii.ToUpper_Chars(Size%3a%206).html>) 11.52 ns 9.81 ns 0.85 0.14 True
[ToLower_Bytes_Chars - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Perf_Ascii.ToLower_Bytes_Chars(Size%3a%206).html>) 11.62 ns 10.49 ns 0.90 0.22 True
[ToUpper_Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Perf_Ascii.ToUpper_Bytes(Size%3a%206).html>) 10.94 ns 9.67 ns 0.88 0.19 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Perf_Ascii.ToLower_Bytes(Size: 6) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.660507012502153 < 10.307724495133526. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.46153271889904 (T) = (0 -9.311200810642834) / Math.Sqrt((0.005453784557512398 / (22)) + (0.06372931391876181 / (13))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .975) and 0.1419338345869719 = (10.851378583562854 - 9.311200810642834) / 10.851378583562854 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Perf_Ascii.ToUpper_Bytes(Size: 128) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.625827246730303 < 18.192140521035835. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 15.38121811226448 (T) = (0 -17.44053809499022) / Math.Sqrt((0.09960943903203688 / (22)) + (0.13276518376382523 / (13))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .975) and 0.09671833286194398 = (19.30797306032852 - 17.44053809499022) / 19.30797306032852 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Perf_Ascii.ToUpper_Bytes_Chars(Size: 6) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.668532271375705 < 11.17785750518028. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.04812772870109 (T) = (0 -10.255282125456596) / Math.Sqrt((0.07843959247220741 / (22)) + (0.04973314370427404 / (13))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .975) and 0.12504547884683676 = (11.720931634184195 - 10.255282125456596) / 11.720931634184195 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Perf_Ascii.ToLower_Chars(Size: 6) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.740393044199786 < 10.820509576317635. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.169675999129595 (T) = (0 -9.689509323143032) / Math.Sqrt((0.10073891334611058 / (22)) + (0.03315291561975875 / (13))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .975) and 0.14313555134310052 = (11.308100526671339 - 9.689509323143032) / 11.308100526671339 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Perf_Ascii.ToLower_Bytes(Size: 128) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.69487799907813 < 18.323266641758412. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.311505403004835 (T) = (0 -17.563117905832392) / Math.Sqrt((0.11067240401833071 / (20)) + (0.10489791167690465 / (13))) is greater than 2.0395134463927453 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (13) - 2, .975) and 0.08678946671034375 = (19.23227696746424 - 17.563117905832392) / 19.23227696746424 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Perf_Ascii.ToUpper_Chars(Size: 6) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.807367196223325 < 10.942521398608795. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.9558758980361 (T) = (0 -9.698291778972221) / Math.Sqrt((0.06070389215479423 / (22)) + (0.041203290361619 / (13))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .975) and 0.15415903047153712 = (11.465857209989247 - 9.698291778972221) / 11.465857209989247 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Perf_Ascii.ToLower_Bytes_Chars(Size: 6) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.48575752764218 < 11.136555533835427. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.321191485826471 (T) = (0 -9.990773090252032) / Math.Sqrt((0.03084622647286224 / (22)) + (0.19181518263637187 / (13))) is greater than 2.0345152974461924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (13) - 2, .975) and 0.14492122989056033 = (11.684038289213209 - 9.990773090252032) / 11.684038289213209 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Perf_Ascii.ToUpper_Bytes(Size: 6) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.670454852578576 < 10.38007405156636. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.39282164442128 (T) = (0 -9.422837876950307) / Math.Sqrt((0.047190261408792594 / (21)) + (0.03372541658115264 / (13))) is greater than 2.036933343456739 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (13) - 2, .975) and 0.13087695000235897 = (10.84177652056965 - 9.422837876950307) / 10.84177652056965 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonFromStream(CollectionsOfPrimitives).JsonNet.html>) 719.63 μs 548.88 μs 0.76 0.02 True
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_FromStream(CollectionsOfPrimitives).SystemTextJsonReflection.html>) 385.31 μs 305.78 μs 0.79 0.08 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.JsonNet_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 548.8808800743724 < 677.9929033109835. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 112.80052082685968 (T) = (0 -548067.0985879991) / Math.Sqrt((86913188.27853628 / (299)) + (26700123.463245843 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2396256111775207 = (720785.8479251772 - 548067.0985879991) / 720785.8479251772 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 305.7849204079687 < 363.75208188667153. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 8.319412042353996 (T) = (0 -319777.4193200875) / Math.Sqrt((55435129.498267725 / (299)) + (744999932.2765427 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16476385430986049 = (382858.6932811218 - 319777.4193200875) / 382858.6932811218 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToDateTime_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Perf_Convert.ToDateTime_String(value%3a%20%2212%2f12%2f1999%22).html>) 215.03 ns 184.70 ns 0.86 0.00 True
[ToDateTime_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Perf_Convert.ToDateTime_String(value%3a%20%22February%2026%2c%202009%22).html>) 269.50 ns 231.31 ns 0.86 0.01 True
[ToDateTime_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Perf_Convert.ToDateTime_String(value%3a%20%22Thursday%2c%20February%2026%2c%202009%22).html>) 412.35 ns 364.63 ns 0.88 0.04 False
[ToDateTime_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Perf_Convert.ToDateTime_String(value%3a%20%22Fri%2c%2027%20Feb%202009%2003%3a11%3a21%20GMT%22).html>) 1.07 μs 974.87 ns 0.91 0.00 False
[ToDateTime_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Perf_Convert.ToDateTime_String(value%3a%20%2212%2f12%2f1999%2011%3a59%3a59%20PM%22).html>) 398.77 ns 356.05 ns 0.89 0.01 False
[GetTypeCode - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Perf_Convert.GetTypeCode.html>) 5.27 ns 4.18 ns 0.79 0.14 False
[ChangeType - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Perf_Convert.ChangeType.html>) 29.88 ns 27.15 ns 0.91 0.09 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Perf_Convert.ToDateTime_String(value: "12/12/1999") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 184.699974459968 < 203.1764535680065. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.91106576308216 (T) = (0 -185.70220639809142) / Math.Sqrt((7.159192250211155 / (299)) + (3.2875451164590976 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15534252229030723 = (219.8550433740633 - 185.70220639809142) / 219.8550433740633 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Perf_Convert.ToDateTime_String(value: "February 26, 2009") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 231.31471550939386 < 256.6361949398678. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 114.10879394346009 (T) = (0 -228.82136505119044) / Math.Sqrt((20.36892336802891 / (299)) + (1.1347401123308127 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16429257847089684 = (273.805591713561 - 228.82136505119044) / 273.805591713561 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Perf_Convert.ToDateTime_String(value: "Thursday, February 26, 2009") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 364.63429179355046 < 390.3636857304481. IsChangePoint: Marked as a change because one of 4/26/2023 8:38:54 PM, 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.25588948492329 (T) = (0 -366.26149676330664) / Math.Sqrt((44.3663575592912 / (299)) + (26.644125408946504 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12177466159444121 = (417.04728928484803 - 366.26149676330664) / 417.04728928484803 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Perf_Convert.ToDateTime_String(value: "Fri, 27 Feb 2009 03:11:21 GMT") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 974.8659592692331 < 1.0158134760212962. IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 51.52354166504218 (T) = (0 -963.2316460006131) / Math.Sqrt((294.00000568026013 / (299)) + (26.31554247547126 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08488967254676652 = (1052.5852644252218 - 963.2316460006131) / 1052.5852644252218 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Perf_Convert.ToDateTime_String(value: "12/12/1999 11:59:59 PM") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 356.04575581600596 < 379.2339139296841. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 35.665077612914786 (T) = (0 -358.2001353480927) / Math.Sqrt((34.71210773392008 / (299)) + (23.030351242898575 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1203361225744453 = (407.2011418684257 - 358.2001353480927) / 407.2011418684257 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Perf_Convert.GetTypeCode ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.18070287439792 < 4.936466214512367. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.116941314839446 (T) = (0 -4.12209293819098) / Math.Sqrt((0.1034084107881195 / (299)) + (0.08573508226253723 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19672004133339988 = (5.131576972284264 - 4.12209293819098) / 5.131576972284264 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Perf_Convert.ChangeType ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.146807351374665 < 28.36536572027584. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.510599492483465 (T) = (0 -26.861722617064178) / Math.Sqrt((1.2625878292208594 / (299)) + (0.45633913159012773 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19355491852026904 = (33.308805812016494 - 26.861722617064178) / 33.308805812016494 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Logging.LoggingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[NoArguments_DefineMessage - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage.html>) 48.44 ns 38.73 ns 0.80 0.17 False
[NoArguments_FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_FilteredByLevel.html>) 25.06 ns 22.11 ns 0.88 0.16 False
[NoArguments_DefineMessage_FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage_FilteredByLevel.html>) 7.73 ns 5.79 ns 0.75 0.21 False
[TwoArguments_DefineMessage_FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_DefineMessage_FilteredByLevel.html>) 8.45 ns 6.12 ns 0.72 0.10 True
[TwoArguments_DefineMessage - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_DefineMessage.html>) 57.56 ns 51.72 ns 0.90 0.34 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.LoggingOverhead*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 38.73293485286207 < 42.68126369158502. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 51.415780872348044 (T) = (0 -38.713986980904345) / Math.Sqrt((1.4682402413031237 / (299)) + (0.1242059842060683 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13772989508165165 = (44.89774927842398 - 38.713986980904345) / 44.89774927842398 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_FilteredByLevel ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.107420796558753 < 23.90699863586643. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.509101461832048 (T) = (0 -22.919927348805324) / Math.Sqrt((0.7812030777635501 / (299)) + (0.13200095464574566 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06243834072657837 = (24.446314673925006 - 22.919927348805324) / 24.446314673925006 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage_FilteredByLevel ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.78585802022667 < 7.595975556850602. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.14757451914103 (T) = (0 -6.008373987575437) / Math.Sqrt((0.063306969145806 / (299)) + (0.036406117816041215 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2533550260497516 = (8.047163239828889 - 6.008373987575437) / 8.047163239828889 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_DefineMessage_FilteredByLevel ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.121021692660198 < 7.982948740488247. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.6125751355141 (T) = (0 -5.892966209242109) / Math.Sqrt((0.038663986508867244 / (299)) + (0.03661662034246862 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31363643162747806 = (8.585779433508216 - 5.892966209242109) / 8.585779433508216 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_DefineMessage ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 51.71862108111195 < 54.91836284110282. IsChangePoint: Marked as a change because one of 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 2.676330043890346 (T) = (0 -54.95584612227369) / Math.Sqrt((13.276770044127014 / (299)) + (22.44010677151334 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.063128244533045 = (58.65887812455466 - 54.95584612227369) / 58.65887812455466 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ArrayList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForNonGeneric(Int32).ArrayList(Size%3a%20512).html>) 1.78 μs 663.70 ns 0.37 0.01 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForNonGeneric&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForNonGeneric<Int32>.ArrayList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 663.6983972715881 < 1.6895215017518876. IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 230.08291778681408 (T) = (0 -662.3672386104419) / Math.Sqrt((6078.052129203388 / (299)) + (0.4987331563256541 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6105346961729203 = (1700.7092341774533 - 662.3672386104419) / 1700.7092341774533 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Compare_Same_Upper - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count%3a%201024%2c%20Options%3a%20(en-US%2c%20Ordinal)).html>) 24.04 ns 19.82 ns 0.82 0.03 False
[Compare_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count%3a%201024%2c%20Options%3a%20(en-US%2c%20Ordinal)).html>) 14.66 ns 9.15 ns 0.62 0.18 True
[Compare_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count%3a%201024%2c%20Options%3a%20(en-US%2c%20OrdinalIgnoreCase)).html>) 13.08 ns 7.69 ns 0.59 0.10 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, Ordinal)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.821664969419935 < 22.98367354653317. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 101.66891946024008 (T) = (0 -19.53168956064314) / Math.Sqrt((0.3720342390629219 / (299)) + (0.016319330941652162 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20650380238881327 = (24.61472357327372 - 19.53168956064314) / 24.61472357327372 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, Ordinal)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.152681524565851 < 13.932680589557389. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.866813916749464 (T) = (0 -9.03650363591255) / Math.Sqrt((0.2672116769403993 / (299)) + (0.13322751024618354 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39913293205769546 = (15.039106181769041 - 9.03650363591255) / 15.039106181769041 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, OrdinalIgnoreCase)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.691596399800671 < 12.703181891026622. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 118.68784487249924 (T) = (0 -7.730789700333075) / Math.Sqrt((0.5862557172759086 / (299)) + (0.004944349297434007 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.42621992652385926 = (13.473437049665234 - 7.730789700333075) / 13.473437049665234 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToWriter(Mode%3a%20Reflection).html>) 386.38 μs 307.59 μs 0.80 0.14 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeObjectProperty(Mode%3a%20Reflection).html>) 407.54 μs 349.24 μs 0.86 0.14 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToString(Mode%3a%20Reflection).html>) 407.57 μs 358.44 μs 0.88 0.16 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToString(Mode%3a%20SourceGen).html>) 359.55 μs 322.75 μs 0.90 0.03 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToStream(Mode%3a%20Reflection).html>) 383.23 μs 322.79 μs 0.84 0.16 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToWriter(Mode%3a%20SourceGen).html>) 319.78 μs 258.57 μs 0.81 0.03 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToStream(Mode%3a%20SourceGen).html>) 369.97 μs 296.55 μs 0.80 0.11 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 333.97 μs 270.67 μs 0.81 0.02 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 394.74 μs 324.81 μs 0.82 0.16 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 307.5932870370371 < 355.67722632092614. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 30.282071410347267 (T) = (0 -306908.9925590346) / Math.Sqrt((111401297.5239724 / (299)) + (89938245.59436274 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21037250659650034 = (388675.6668466255 - 306908.9925590346) / 388675.6668466255 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 349.2355324074074 < 404.33743996888523. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.788147189413595 (T) = (0 -359336.95418863336) / Math.Sqrt((137669364.45578572 / (299)) + (346377114.8065503 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16650597803840417 = (431121.21349466644 - 359336.95418863336) / 431121.21349466644 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 358.4448425925925 < 410.14428744369366. IsChangePoint: Marked as a change because one of 3/13/2023 9:17:14 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 27.676806290947752 (T) = (0 -350996.4807769228) / Math.Sqrt((199664753.6004906 / (299)) + (95215320.3336268 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18228253241303666 = (429239.30903994624 - 350996.4807769228) / 429239.30903994624 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 322.7498263888889 < 343.1529140625. IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.71097939955677 (T) = (0 -309086.85743419745) / Math.Sqrt((74622505.31672204 / (299)) + (164096174.53743914 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16246297609213098 = (369042.6197423813 - 309086.85743419745) / 369042.6197423813 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 322.7941239316239 < 367.28067721861476. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 26.36781852801996 (T) = (0 -317079.73837274883) / Math.Sqrt((141345935.62664822 / (299)) + (106199128.57629508 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19643946473124124 = (394593.4680163186 - 317079.73837274883) / 394593.4680163186 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 258.5700068306011 < 305.9123829964678. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.77036852833719 (T) = (0 -274217.28493789444) / Math.Sqrt((66165901.17713539 / (299)) + (159265355.16054097 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17762158733478806 = (333444.1672042376 - 274217.28493789444) / 333444.1672042376 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 296.551488095238 < 359.5847777517198. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.481638503121406 (T) = (0 -312820.7938408902) / Math.Sqrt((125164577.28348519 / (299)) + (133584404.07014894 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20377748326601974 = (392881.1196197361 - 312820.7938408902) / 392881.1196197361 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 270.6728679187192 < 320.0961068104104. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.328476702251184 (T) = (0 -279508.628051933) / Math.Sqrt((68461543.9284286 / (299)) + (127891491.15181036 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18749189866786656 = (344007.18908976973 - 279508.628051933) / 344007.18908976973 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 324.80545068027203 < 375.77860184151785. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 26.969298227046856 (T) = (0 -313673.73500991316) / Math.Sqrt((171085586.71073684 / (299)) + (120106569.25014138 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21217007534804008 = (398149.04866489425 - 313673.73500991316) / 398149.04866489425 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractSerializerBinaryXml - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Xml_ToStream(LoginViewModel).DataContractSerializerBinaryXml.html>) 748.20 ns 528.85 ns 0.71 0.04 False
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlToStream(LoginViewModel).DataContractSerializer.html>) 1.33 μs 1.03 μs 0.78 0.27 False
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlToStream(LoginViewModel).XmlSerializer.html>) 2.18 μs 1.95 μs 0.90 0.12 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.DataContractSerializer_BinaryXml_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 528.8512928881056 < 692.8422849503958. IsChangePoint: Marked as a change because one of 4/4/2023 3:37:20 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 59.40624003235564 (T) = (0 -508.74135242829016) / Math.Sqrt((2123.9375374869473 / (218)) + (146.76081755908834 / (13))) is greater than 1.9703772833260038 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (218) + (13) - 2, .975) and 0.3487553014281688 = (781.183099906919 - 508.74135242829016) / 781.183099906919 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.DataContractSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0314611922876984 < 1.2245351547298735. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.35040269794062 (T) = (0 -985.276648540561) / Math.Sqrt((2289.3541470143714 / (299)) + (1235.4553774492856 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2610645415939015 = (1333.3730806014187 - 985.276648540561) / 1333.3730806014187 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.XmlSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.9530112559665336 < 2.0761536959964157. IsChangePoint: Marked as a change because one of 3/1/2023 1:48:25 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.434183082960338 (T) = (0 -1995.5531311700786) / Math.Sqrt((4042.2923251244947 / (299)) + (2514.121857814252 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11729284396126001 = (2260.719330888146 - 1995.5531311700786) / 2260.719330888146 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).SortedSet(Size%3a%20512).html>) 10.36 μs 5.84 μs 0.56 0.02 True
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).HashSet(Size%3a%20512).html>) 1.98 μs 819.34 ns 0.41 0.01 True
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).SortedList(Size%3a%20512).html>) 3.12 μs 1.87 μs 0.60 0.00 True
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).ImmutableDictionary(Size%3a%20512).html>) 16.49 μs 12.98 μs 0.79 0.01 True
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).Queue(Size%3a%20512).html>) 2.36 μs 1.79 μs 0.76 0.01 True
[ImmutableQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).ImmutableQueue(Size%3a%20512).html>) 4.00 μs 733.63 ns 0.18 0.08 True
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).Stack(Size%3a%20512).html>) 2.04 μs 1.50 μs 0.73 0.01 True
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).SortedDictionary(Size%3a%20512).html>) 10.79 μs 8.83 μs 0.82 0.03 False
[ConcurrentQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).ConcurrentQueue(Size%3a%20512).html>) 7.27 μs 4.88 μs 0.67 0.13 True
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).ImmutableSortedSet(Size%3a%20512).html>) 10.29 μs 5.36 μs 0.52 0.13 True
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).Dictionary(Size%3a%20512).html>) 2.04 μs 1.27 μs 0.62 0.02 True
[IEnumerable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).IEnumerable(Size%3a%20512).html>) 3.37 μs 912.68 ns 0.27 0.08 True
[ConcurrentBag - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).ConcurrentBag(Size%3a%20512).html>) 2.53 μs 1.66 μs 0.66 0.00 True
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 7.43 μs 4.83 μs 0.65 0.00 True
[ConcurrentStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).ConcurrentStack(Size%3a%20512).html>) 3.12 μs 1.75 μs 0.56 0.12 True
[ImmutableStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).ImmutableStack(Size%3a%20512).html>) 2.09 μs 695.73 ns 0.33 0.01 True
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).LinkedList(Size%3a%20512).html>) 2.32 μs 1.72 μs 0.74 0.15 False
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(Int32).ImmutableList(Size%3a%20512).html>) 8.32 μs 5.02 μs 0.60 0.00 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.838150020234093 < 9.725852401863444. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 248.7874223710397 (T) = (0 -5854.694075193922) / Math.Sqrt((59341.302774196156 / (299)) + (1332.669193486875 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4243649330455581 = (10170.843319483325 - 5854.694075193922) / 10170.843319483325 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.HashSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 819.3420916349668 < 1.8890597097118607. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 720.9477305843122 (T) = (0 -818.9829146120453) / Math.Sqrt((791.6715503264071 / (299)) + (0.5919889754325791 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5909468982508366 = (2002.1432696878953 - 818.9829146120453) / 2002.1432696878953 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.8688089369040375 < 2.966183925034553. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 273.4338764075544 (T) = (0 -1891.9536794291598) / Math.Sqrt((210.7177267878927 / (299)) + (260.2547976629878 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39683960338153024 = (3136.7339268892993 - 1891.9536794291598) / 3136.7339268892993 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.979558522872473 < 15.62929236458814. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.101037526451265 (T) = (0 -13680.72334268523) / Math.Sqrt((43608.93252115972 / (299)) + (205566.27672800864 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1758113773409051 = (16599.019892493616 - 13680.72334268523) / 16599.019892493616 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7870308391993894 < 2.2258827097283076. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 70.23696505737308 (T) = (0 -1753.8407136320932) / Math.Sqrt((1549.011412656342 / (299)) + (778.938301591983 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24421014865162305 = (2320.540174630726 - 1753.8407136320932) / 2320.540174630726 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.ImmutableQueue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 733.6276277617012 < 3.799460803689589. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 249.2174262744608 (T) = (0 -730.2605029309238) / Math.Sqrt((3425.3077836166926 / (299)) + (2075.6594482514506 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.816993988879362 = (3990.3634774572215 - 730.2605029309238) / 3990.3634774572215 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.4953929895920564 < 1.9405666907139754. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 44.910339945129856 (T) = (0 -1548.5276908881267) / Math.Sqrt((645.4400815528296 / (299)) + (1437.823687456953 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23545540144232407 = (2025.424931141291 - 1548.5276908881267) / 2025.424931141291 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.834052944507363 < 10.436967681665699. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 130.35935534695022 (T) = (0 -8832.804191952695) / Math.Sqrt((70721.76740708356 / (299)) + (144.08872186433018 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18846692402698487 = (10884.096352280287 - 8832.804191952695) / 10884.096352280287 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.ConcurrentQueue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.884572560043928 < 7.181383676518909. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 133.56596665681693 (T) = (0 -4784.906025911674) / Math.Sqrt((49521.42221368927 / (299)) + (3609.277276970519 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3701559842389068 = (7596.969894410558 - 4784.906025911674) / 7596.969894410558 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.364559273097826 < 9.734126571518944. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 220.78151203846724 (T) = (0 -5352.163272372092) / Math.Sqrt((87018.39031578637 / (299)) + (2524.543167158361 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.47607476475598276 = (10215.509603921506 - 5352.163272372092) / 10215.509603921506 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2661221005685563 < 1.9293866292599546. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 296.30252402777535 (T) = (0 -1272.0209094921895) / Math.Sqrt((1456.2463388021933 / (299)) + (38.117931640275735 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.39418443162608974 = (2099.6834282526993 - 1272.0209094921895) / 2099.6834282526993 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.IEnumerable(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 912.6840334097521 < 3.2026831414565664. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 418.9745381124583 (T) = (0 -917.2254321334996) / Math.Sqrt((9983.202767133209 / (299)) + (12.993761373117145 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7281611593783696 = (3374.1515010732983 - 917.2254321334996) / 3374.1515010732983 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.ConcurrentBag(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.663684967078691 < 2.406105768881907. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 795.1026554031989 (T) = (0 -1659.8305015460683) / Math.Sqrt((232.27835103014186 / (299)) + (5.700322166702616 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34558821976946424 = (2536.3701444392464 - 1659.8305015460683) / 2536.3701444392464 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.834045754941652 < 7.082853067275461. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 493.13997601123344 (T) = (0 -4841.966334958272) / Math.Sqrt((3121.373649615068 / (299)) + (248.71946135488955 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3564340734726232 = (7523.652411315623 - 4841.966334958272) / 7523.652411315623 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.ConcurrentStack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7517496803151922 < 2.9711170738716537. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 335.7047437876131 (T) = (0 -1741.7644450541143) / Math.Sqrt((4207.779892603792 / (299)) + (39.34186979148233 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4435151948251 = (3129.9407079168814 - 1741.7644450541143) / 3129.9407079168814 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.ImmutableStack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 695.7297804867077 < 1.960923416213215. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 452.72458227850086 (T) = (0 -693.5108877432805) / Math.Sqrt((467.51089569307953 / (299)) + (99.42847816998936 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6645788205322035 = (2067.5822822030946 - 693.5108877432805) / 2067.5822822030946 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7245906452483801 < 2.2151134294402417. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 59.901360688016005 (T) = (0 -1708.2149255200286) / Math.Sqrt((13222.723003733621 / (299)) + (859.32171013645 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26917928978348776 = (2337.3926075712257 - 1708.2149255200286) / 2337.3926075712257 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<Int32>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.022024720406848 < 7.655359919043099. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 481.57246144819413 (T) = (0 -5011.771152674717) / Math.Sqrt((9688.808724339302 / (299)) + (122.48071482540809 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.38326016868471346 = (8126.232323906165 - 5011.771152674717) / 8126.232323906165 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetDateTimeOffset - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Get.GetDateTimeOffset.html>) 6.60 μs 6.19 μs 0.94 0.00 True
[GetBoolean - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Get.GetBoolean.html>) 129.52 ns 121.32 ns 0.94 0.03 False
[GetSingle - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Get.GetSingle.html>) 5.50 μs 5.18 μs 0.94 0.00 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Get.GetDateTimeOffset ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.194850424200733 < 6.263223717843617. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.045358231183855 (T) = (0 -6145.904623747167) / Math.Sqrt((1204.2011256887101 / (299)) + (1743.923794998317 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0727900010662182 = (6628.384757298208 - 6145.904623747167) / 6628.384757298208 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Get.GetBoolean ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 121.31513887398874 < 123.01584241631836. IsChangePoint: Marked as a change because one of 3/29/2023 12:02:44 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 53.48121543704253 (T) = (0 -121.35496828355687) / Math.Sqrt((4.490563157758481 / (299)) + (0.4382470813564705 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08865893238017893 = (133.1608687409463 - 121.35496828355687) / 133.1608687409463 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Get.GetSingle ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.1762223463533035 < 5.254619230014801. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 62.79768388202535 (T) = (0 -5194.5516951366235) / Math.Sqrt((1156.331067463162 / (299)) + (356.4686136364235 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06333840235767375 = (5545.8147405763675 - 5194.5516951366235) / 5545.8147405763675 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_DateTimes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteDateTimes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted%3a%20False%2c%20SkipValidation%3a%20True).html>) 5.42 ms 5.05 ms 0.93 0.00 True
[WriteDateTimes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 6.26 ms 5.68 ms 0.91 0.00 True
[WriteDateTimes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted%3a%20False%2c%20SkipValidation%3a%20False).html>) 5.50 ms 5.08 ms 0.92 0.00 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_DateTimes*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: False, SkipValidation: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.047723469387756 < 5.141340585662526. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 90.91696945235061 (T) = (0 -5053768.639391498) / Math.Sqrt((2256125772.8071294 / (299)) + (171353800.67701447 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07570177938112799 = (5467681.887354173 - 5053768.639391498) / 5467681.887354173 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.677122719141324 < 5.951689320054945. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.28896520923737 (T) = (0 -5714285.621490167) / Math.Sqrt((2619786100.562193 / (299)) + (1693340205.0362637 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09231256014775523 = (6295433.175125075 - 5714285.621490167) / 6295433.175125075 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: False, SkipValidation: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.084888839285714 < 5.281479731060607. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 82.20291317129109 (T) = (0 -5074197.60074518) / Math.Sqrt((2152024429.5200953 / (299)) + (433387336.988323 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09349815557841167 = (5597559.047420221 - 5074197.60074518) / 5597559.047420221 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_SortedSet

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EnumerateViewBetween - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_SortedSet.EnumerateViewBetween.html>) 3.11 μs 1.86 μs 0.60 0.08 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_SortedSet*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_SortedSet.EnumerateViewBetween ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.860075514908394 < 2.9506595412564836. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 188.0149976934293 (T) = (0 -1867.4166562150094) / Math.Sqrt((8544.457883883893 / (299)) + (243.17537392584038 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4090924077550107 = (3160.2515870887332 - 1867.4166562150094) / 3160.2515870887332 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).ImmutableSortedSet(Size%3a%20512).html>) 32.34 μs 26.69 μs 0.83 0.14 False
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).ImmutableList(Size%3a%20512).html>) 1.38 ms 1.28 ms 0.93 0.10 False
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).SortedSet(Size%3a%20512).html>) 31.26 μs 21.12 μs 0.68 0.12 False
[ImmutableHashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsFalse(Int32).ImmutableHashSet(Size%3a%20512).html>) 18.55 μs 15.23 μs 0.82 0.05 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 26.68603398370498 < 30.722875910686977. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 2:01:50 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 44.25718728601033 (T) = (0 -26998.88971525849) / Math.Sqrt((2574668.4617939955 / (299)) + (30781.52838064545 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14658065589162553 = (31636.135156352797 - 26998.88971525849) / 31636.135156352797 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsFalse<Int32>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2818410256410255 < 1.3135754395604395. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 30.506846926575154 (T) = (0 -1280139.5478344387) / Math.Sqrt((1283996268.6119912 / (299)) + (94706533.22224683 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.075010204626074 = (1383949.9140819644 - 1280139.5478344387) / 1383949.9140819644 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsFalse<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 21.115133626522326 < 29.664644350709295. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.03849624268163 (T) = (0 -21244.375243864295) / Math.Sqrt((7368937.410880574 / (299)) + (11393.33440464717 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3250364984230959 = (31474.850409290982 - 21244.375243864295) / 31474.850409290982 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsFalse<Int32>.ImmutableHashSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.228239917976758 < 17.59728665992143. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.547637352238034 (T) = (0 -14645.936663203793) / Math.Sqrt((128137.07291032352 / (299)) + (133143.5937125983 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20494187812638656 = (18421.21507882915 - 14645.936663203793) / 18421.21507882915 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Numerics.Tests.Perf_VectorOf(Single).GetHashCodeBenchmark.html>) 23.69 ns 19.52 ns 0.82 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Single>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.520907873769932 < 22.433976204120384. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 85.29890991721429 (T) = (0 -19.710137308773486) / Math.Sqrt((0.1438125509387317 / (299)) + (0.01517892009375024 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1494540346549547 = (23.173512204925427 - 19.710137308773486) / 23.173512204925427 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndRemove(Int32).SortedSet(Size%3a%20512).html>) 119.18 μs 100.45 μs 0.84 0.12 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndRemove(Int32).SortedDictionary(Size%3a%20512).html>) 143.44 μs 111.86 μs 0.78 0.11 False
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndRemove(Int32).Dictionary(Size%3a%20512).html>) 17.03 μs 14.15 μs 0.83 0.01 False
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndRemove(Int32).HashSet(Size%3a%20512).html>) 16.03 μs 13.49 μs 0.84 0.01 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndRemove(Int32).SortedList(Size%3a%20512).html>) 144.53 μs 134.04 μs 0.93 0.00 True
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndRemove(Int32).Queue(Size%3a%20512).html>) 3.62 μs 2.98 μs 0.82 0.00 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndRemove<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 100.45228110599078 < 113.13939597790886. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 52.151676409792074 (T) = (0 -100809.49291237893) / Math.Sqrt((15828259.477588957 / (299)) + (848616.4334599049 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15101027757172636 = (118740.53389485614 - 100809.49291237893) / 118740.53389485614 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndRemove<Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 111.86108516483519 < 135.9354642038008. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 105.29569627088746 (T) = (0 -113007.95243517646) / Math.Sqrt((15306012.864876164 / (299)) + (309521.48770959827 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20329617858457777 = (141844.37101658026 - 113007.95243517646) / 141844.37101658026 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndRemove<Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.149175547235021 < 16.19271031866334. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.60785947318951 (T) = (0 -14121.376141175131) / Math.Sqrt((1123528.3211740672 / (299)) + (165551.24126880188 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2648147419621875 = (19207.915265962572 - 14121.376141175131) / 19207.915265962572 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndRemove<Int32>.HashSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.493754173344017 < 15.197454484494415. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 59.88209878620879 (T) = (0 -13516.581820791092) / Math.Sqrt((2129327.722349593 / (299)) + (8676.885169616162 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28108940707571833 = (18801.478172425133 - 13516.581820791092) / 18801.478172425133 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndRemove<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 134.04109748010612 < 136.8841708715596. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.852475468096763 (T) = (0 -134430.38658168225) / Math.Sqrt((401506.56884304393 / (299)) + (1333574.544124483 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06471300483899824 = (143731.69655645773 - 134430.38658168225) / 143731.69655645773 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndRemove<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.980581477443994 < 3.4424263948336433. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 159.1488545443501 (T) = (0 -2954.365252440032) / Math.Sqrt((257.37864524776165 / (299)) + (236.11305922209598 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1902531996181578 = (3648.5050031032893 - 2954.365252440032) / 3648.5050031032893 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToWriter(Mode%3a%20SourceGen).html>) 531.10 ns 474.56 ns 0.89 0.00 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 656.39 ns 582.13 ns 0.89 0.00 True
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 838.09 ns 696.81 ns 0.83 0.01 True
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 976.53 ns 899.51 ns 0.92 0.12 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeObjectProperty(Mode%3a%20Reflection).html>) 1.10 μs 998.05 ns 0.90 0.03 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToStream(Mode%3a%20SourceGen).html>) 715.16 ns 638.29 ns 0.89 0.00 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToString(Mode%3a%20Reflection).html>) 893.08 ns 779.19 ns 0.87 0.00 True
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToString(Mode%3a%20SourceGen).html>) 712.21 ns 642.75 ns 0.90 0.00 True
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToStream(Mode%3a%20Reflection).html>) 950.32 ns 856.00 ns 0.90 0.02 True
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToWriter(Mode%3a%20Reflection).html>) 722.34 ns 597.57 ns 0.83 0.01 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 474.55637992714264 < 505.5747606291308. IsChangePoint: Marked as a change because one of 3/19/2023 10:05:03 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 54.803752293959775 (T) = (0 -476.69198505403773) / Math.Sqrt((63.69432921508165 / (299)) + (12.881887200781248 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11201594293115416 = (536.8249365056782 - 476.69198505403773) / 536.8249365056782 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 582.1319564414567 < 623.6168052300359. IsChangePoint: Marked as a change because one of 3/19/2023 10:05:03 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 140.7377310966752 (T) = (0 -581.1714287908728) / Math.Sqrt((58.0499300605597 / (299)) + (2.44928456888065 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1302680111682682 = (668.2189872900177 - 581.1714287908728) / 668.2189872900177 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 696.8086123511906 < 791.9496969673373. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.11664634196853 (T) = (0 -705.1827518926425) / Math.Sqrt((147.67760302223812 / (299)) + (105.78916034127899 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16408812282048038 = (843.6089630309178 - 705.1827518926425) / 843.6089630309178 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 899.5132533241236 < 918.8446030147463. IsChangePoint: Marked as a change because one of 2/5/2023 4:54:05 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 15.157725948053866 (T) = (0 -863.2834234306005) / Math.Sqrt((511.5122137291513 / (299)) + (689.9621542852238 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11501311028407042 = (975.4759459857132 - 863.2834234306005) / 975.4759459857132 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 998.0503507871322 < 1.0471071018275002. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.348032079564565 (T) = (0 -970.4653797842432) / Math.Sqrt((409.4971740779753 / (299)) + (625.6918432734379 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13402462878846358 = (1120.6616400955154 - 970.4653797842432) / 1120.6616400955154 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 638.2938766442359 < 673.3645032489648. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.880569843601926 (T) = (0 -646.2355990048553) / Math.Sqrt((97.6925807258031 / (299)) + (58.463962144565556 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1167192072542851 = (731.630987917223 - 646.2355990048553) / 731.630987917223 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 779.1867107951502 < 844.41066634084. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 45.73993841818517 (T) = (0 -766.609016134086) / Math.Sqrt((138.08121622940658 / (299)) + (107.69042714806051 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14998424625237336 = (901.8762449450973 - 766.609016134086) / 901.8762449450973 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 642.7523947688278 < 679.8954549035922. IsChangePoint: Marked as a change because one of 3/20/2023 7:45:43 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 141.95414434257495 (T) = (0 -640.2353690591904) / Math.Sqrt((71.6987245282852 / (299)) + (1.5156254426900742 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11689083602963786 = (724.9787400922919 - 640.2353690591904) / 724.9787400922919 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 855.9970312471074 < 900.2242583675566. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.81850207271093 (T) = (0 -812.963406060824) / Math.Sqrt((198.44440566920065 / (299)) + (531.6392807271151 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15322117491257162 = (960.0658188127071 - 812.963406060824) / 960.0658188127071 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 597.5699628651276 < 685.9762002458633. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 46.99880865508106 (T) = (0 -599.8211361979617) / Math.Sqrt((99.51523510386022 / (299)) + (101.27894625968838 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18255564884173198 = (733.7761100777714 - 599.8211361979617) / 733.7761100777714 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[StaticMethod4_ByRefParams_int_string_struct_class - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Reflection.Invoke.StaticMethod4_ByRefParams_int_string_struct_class.html>) 272.73 ns 246.56 ns 0.90 0.17 False
[Property_Set_class - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Reflection.Invoke.Property_Set_class.html>) 44.24 ns 35.73 ns 0.81 0.03 False
[Property_Set_int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Reflection.Invoke.Property_Set_int.html>) 51.58 ns 42.59 ns 0.83 0.22 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Reflection.Invoke.StaticMethod4_ByRefParams_int_string_struct_class ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 246.55514346764346 < 262.3988172504579. IsChangePoint: Marked as a change because one of 5/1/2023 3:45:46 PM, 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.80525112863012 (T) = (0 -257.8470022108795) / Math.Sqrt((149.45826710075437 / (299)) + (49.29631550048083 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1063079206368936 = (288.5188401743868 - 257.8470022108795) / 288.5188401743868 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Reflection.Invoke.Property_Set_class ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 35.73332763857252 < 39.77387535816446. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 43.64614063272959 (T) = (0 -35.894858794149826) / Math.Sqrt((1.0966664575188254 / (299)) + (0.07403912785369092 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1052723350931183 = (40.11819484522764 - 35.894858794149826) / 40.11819484522764 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Reflection.Invoke.Property_Set_int ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 42.589144883824844 < 47.474101679693554. IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.08266670679522 (T) = (0 -42.14394948734119) / Math.Sqrt((2.504030102247507 / (299)) + (0.9877529353391172 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09977692784595081 = (46.8150070698581 - 42.14394948734119) / 46.8150070698581 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DateTime_ToString_Unspecified - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified.html>) 83.36 ns 73.67 ns 0.88 0.00 True
[DateTime_ToString_Local - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Local.html>) 222.25 ns 183.95 ns 0.83 0.01 False
[DateTime_ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Tests.Perf_XmlConvert.DateTime_ToString.html>) 87.65 ns 78.73 ns 0.90 0.01 False
[DateTime_ToString_RoundtripKind - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_RoundtripKind.html>) 84.23 ns 73.32 ns 0.87 0.00 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 73.67039307161645 < 78.8870840537771. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.07048906240553 (T) = (0 -73.84193397061298) / Math.Sqrt((0.7874997803044758 / (299)) + (0.6762267357796251 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11257757953104638 = (83.2094527559847 - 73.84193397061298) / 83.2094527559847 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Local ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 183.94716075723792 < 210.3555559312076. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.7283416838846 (T) = (0 -185.0624372876129) / Math.Sqrt((29.605473389561478 / (299)) + (0.4175643499960715 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12905910589307765 = (212.4856445940331 - 185.0624372876129) / 212.4856445940331 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 78.73497368509614 < 83.70697624835655. IsChangePoint: Marked as a change because one of 3/31/2023 7:02:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 43.09731123785088 (T) = (0 -79.1075914489941) / Math.Sqrt((1.3589546817110014 / (299)) + (0.4638474833723034 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09850258505244622 = (87.75132367251028 - 79.1075914489941) / 87.75132367251028 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_RoundtripKind ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 73.32224404809327 < 79.26865628301967. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 54.361392729612184 (T) = (0 -73.5024010265006) / Math.Sqrt((0.9094375192149866 / (299)) + (0.3126696542613357 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10852451978816753 = (82.45027783493828 - 73.5024010265006) / 82.45027783493828 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_UInt16.TryParse(value%3a%20%2212345%22).html>) 14.04 ns 11.56 ns 0.82 0.10 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_UInt16.Parse(value%3a%20%2212345%22).html>) 14.44 ns 11.93 ns 0.83 0.13 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_UInt16.TryParse(value%3a%20%2265535%22).html>) 14.26 ns 11.59 ns 0.81 0.14 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_UInt16.ToString(value%3a%2012345).html>) 12.12 ns 10.80 ns 0.89 0.30 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_UInt16.TryParse(value: "12345") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.561566330153948 < 13.173296356076621. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 46.514808642474456 (T) = (0 -11.528206340432487) / Math.Sqrt((1.7614527248298995 / (299)) + (0.014709217017821581 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2526865034330121 = (15.426198500884585 - 11.528206340432487) / 15.426198500884585 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_UInt16.Parse(value: "12345") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.932585900888238 < 13.72635816009631. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.519253410946284 (T) = (0 -11.687879300789287) / Math.Sqrt((1.7457031594024304 / (299)) + (0.27650868862010636 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24886228925710563 = (15.560235005681816 - 11.687879300789287) / 15.560235005681816 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_UInt16.TryParse(value: "65535") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.590824062973931 < 13.538279648129748. IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.063508450729145 (T) = (0 -11.49138687519034) / Math.Sqrt((1.8508338529560575 / (299)) + (0.0735995209821989 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2598762421915549 = (15.526304559141689 - 11.49138687519034) / 15.526304559141689 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_UInt16.ToString(value: 12345) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.797752981690733 < 11.603568889716467. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.124896160493233 (T) = (0 -11.032311172940476) / Math.Sqrt((0.3055826976661465 / (299)) + (0.10401812895107893 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12191381447584632 = (12.564041383198607 - 11.032311172940476) / 12.564041383198607 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeToString(Mode%3a%20SourceGen).html>) 10.44 μs 9.60 μs 0.92 0.00 True
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeToString(Mode%3a%20Reflection).html>) 11.46 μs 10.05 μs 0.88 0.00 True
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 10.38 μs 8.94 μs 0.86 0.00 True
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 9.35 μs 8.42 μs 0.90 0.00 True
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeObjectProperty(Mode%3a%20Reflection).html>) 11.65 μs 10.34 μs 0.89 0.00 True
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeToStream(Mode%3a%20SourceGen).html>) 8.89 μs 8.00 μs 0.90 0.00 True
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeToWriter(Mode%3a%20Reflection).html>) 9.56 μs 8.20 μs 0.86 0.00 True
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 10.64 μs 9.75 μs 0.92 0.00 True
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Dictionary(String%2c%20String)).SerializeToStream(Mode%3a%20Reflection).html>) 10.67 μs 9.03 μs 0.85 0.00 True

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.602284059444495 < 9.910392534112507. IsChangePoint: Marked as a change because one of 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 91.93305343104835 (T) = (0 -9512.475910457139) / Math.Sqrt((14794.25636770138 / (299)) + (1453.161881233951 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10931206338214562 = (10679.920002709572 - 9512.475910457139) / 10679.920002709572 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.04774008063777 < 10.887558680977438. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 95.55162339952538 (T) = (0 -10111.448580616807) / Math.Sqrt((12442.90038758105 / (299)) + (3000.3173331336134 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1349240614536382 = (11688.509794420674 - 10111.448580616807) / 11688.509794420674 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.937050750800035 < 9.789193370389398. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 111.09708723673687 (T) = (0 -9015.118270355182) / Math.Sqrt((11713.822744729237 / (299)) + (2061.4923066718115 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14770138352914813 = (10577.417463945272 - 9015.118270355182) / 10577.417463945272 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.416747305125059 < 8.882654387701598. IsChangePoint: Marked as a change because one of 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 128.6515688813507 (T) = (0 -8395.411076804718) / Math.Sqrt((12568.643480873854 / (299)) + (554.6347373819548 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12359955864519684 = (9579.423606663735 - 8395.411076804718) / 9579.423606663735 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.342723754931473 < 11.062781137010917. IsChangePoint: Marked as a change because one of 5/4/2023 7:16:27 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 136.94416022153675 (T) = (0 -10271.208150772476) / Math.Sqrt((17484.997739345104 / (299)) + (1199.3225720488788 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14066606105965426 = (11952.52239593611 - 10271.208150772476) / 11952.52239593611 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.998520622589331 < 8.470494242198466. IsChangePoint: Marked as a change because one of 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 95.54807567409064 (T) = (0 -8025.43736959448) / Math.Sqrt((12543.409043348705 / (299)) + (1425.1550520463327 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12784037434798046 = (9201.798768883307 - 8025.43736959448) / 9201.798768883307 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.20100968395766 < 9.150450508342022. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 134.81082684446 (T) = (0 -8246.650192090296) / Math.Sqrt((12765.028525512847 / (299)) + (1398.2286272021663 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16692977694053993 = (9899.10569820198 - 8246.650192090296) / 9899.10569820198 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.745275923453491 < 10.132324347303946. IsChangePoint: Marked as a change because one of 4/7/2023 7:05:26 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 116.56911270579849 (T) = (0 -9752.782540700002) / Math.Sqrt((17918.615394664877 / (299)) + (694.8536483831746 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11289987066160333 = (10994.004192033735 - 9752.782540700002) / 10994.004192033735 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.031135537790696 < 10.142668743688546. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 170.98216151021845 (T) = (0 -9034.434713830902) / Math.Sqrt((10772.08641211511 / (299)) + (1054.8767847693332 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17002918520883065 = (10885.243857766342 - 9034.434713830902) / 10885.243857766342 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.ComponentModel.Tests.Perf_TypeDescriptorTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.ComponentModel.Tests.SomeValueType%3f)).html>) 173.79 ns 156.39 ns 0.90 0.01 True
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.ComponentModel.Tests.ClassWithNoConverter)).html>) 118.59 ns 105.52 ns 0.89 0.01 False
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(string)).html>) 116.59 ns 106.36 ns 0.91 0.03 False
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.ComponentModel.Tests.ClassIDerived)).html>) 116.68 ns 105.73 ns 0.91 0.00 True
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.Guid)).html>) 177.78 ns 157.31 ns 0.88 0.01 False
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.ComponentModel.Tests.IDerived)).html>) 100.90 ns 88.16 ns 0.87 0.02 True
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(int)).html>) 173.62 ns 157.78 ns 0.91 0.02 False
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(int%3f)).html>) 175.28 ns 156.07 ns 0.89 0.01 True
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.ComponentModel.Tests.SomeEnum)).html>) 220.05 ns 195.11 ns 0.89 0.07 False
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.Enum)).html>) 173.44 ns 158.40 ns 0.91 0.03 False
[GetConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert%3a%20typeof(System.ComponentModel.Tests.DerivedClass)).html>) 174.52 ns 156.57 ns 0.90 0.03 False

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeValueType?)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 156.38514924626062 < 165.18085083461813. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.22011537997703 (T) = (0 -158.50648878539397) / Math.Sqrt((2.95972132962702 / (299)) + (11.23818365327339 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09221869316796548 = (174.60867236685917 - 158.50648878539397) / 174.60867236685917 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassWithNoConverter)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 105.52020928118613 < 112.41333075393645. IsChangePoint: Marked as a change because one of 1/31/2023 1:54:46 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 51.27774633135533 (T) = (0 -105.53205993460443) / Math.Sqrt((1.4821903280971858 / (299)) + (0.6468153687886833 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10205542197793974 = (117.52625108229316 - 105.53205993460443) / 117.52625108229316 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(string)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 106.36351125357118 < 110.90004927763864. IsChangePoint: Marked as a change because one of 1/31/2023 1:54:46 PM, 2/9/2023 1:41:04 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.82126714735859 (T) = (0 -104.76691109317841) / Math.Sqrt((2.639977553807904 / (299)) + (1.0532792283844106 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10458033529706064 = (117.00313855395999 - 104.76691109317841) / 117.00313855395999 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.ClassIDerived)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 105.73136577099515 < 111.18474631894905. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 30.606912670203474 (T) = (0 -105.6056071833348) / Math.Sqrt((1.1558594179473385 / (299)) + (1.9208477986984451 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10140921932181932 = (117.52358187297735 - 105.6056071833348) / 117.52358187297735 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Guid)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 157.3086516780102 < 165.6203792709481. IsChangePoint: Marked as a change because one of 1/31/2023 1:54:46 PM, 2/9/2023 1:41:04 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 53.35553410730969 (T) = (0 -158.1341688688928) / Math.Sqrt((3.37908498777201 / (299)) + (1.088433222855006 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0942114925784044 = (174.58177882940382 - 158.1341688688928) / 174.58177882940382 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.IDerived)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 88.15600712647482 < 95.30835565298122. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.349566745100375 (T) = (0 -86.67235314444866) / Math.Sqrt((4.324439963491482 / (299)) + (1.3929223732227296 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1336741340878447 = (100.04590253482881 - 86.67235314444866) / 100.04590253482881 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(int)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 157.78311878362595 < 164.1196958002706. IsChangePoint: Marked as a change because one of 5/4/2023 7:16:27 PM, 5/8/2023 9:53:14 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.845275836075864 (T) = (0 -156.43024121064965) / Math.Sqrt((8.586470912024431 / (299)) + (1.5322739691514908 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10678316952255024 = (175.13131848067982 - 156.43024121064965) / 175.13131848067982 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(int?)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 156.07351827452774 < 166.035978133159. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 35.08583317549384 (T) = (0 -157.64116320132) / Math.Sqrt((3.5046054695396687 / (299)) + (3.00201274033754 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09880250233378009 = (174.92410221905232 - 157.64116320132) / 174.92410221905232 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.SomeEnum)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 195.10742179316355 < 205.9417715127455. IsChangePoint: Marked as a change because one of 1/31/2023 7:06:58 PM, 2/9/2023 1:41:04 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.942555088734764 (T) = (0 -198.5865835571293) / Math.Sqrt((5.175814218067226 / (299)) + (31.18415522447839 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09198554322636901 = (218.70420903071278 - 198.5865835571293) / 218.70420903071278 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.Enum)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 158.39979501243852 < 164.01879521396626. IsChangePoint: Marked as a change because one of 1/31/2023 1:54:46 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.20959095826889 (T) = (0 -156.26826918950542) / Math.Sqrt((11.217960231709412 / (299)) + (2.38308624923073 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10547413966364645 = (174.69396483489754 - 156.26826918950542) / 174.69396483489754 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.DerivedClass)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 156.56842568620593 < 163.87665995040206. IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 52.45240834960932 (T) = (0 -157.19870080956335) / Math.Sqrt((9.77086803632648 / (299)) + (1.179275837487997 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10491212470036292 = (175.62376292600317 - 157.19870080956335) / 175.62376292600317 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dequeue_And_Enqueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Int32%2c%20Int32).Dequeue_And_Enqueue(Size%3a%201000).html>) 199.71 μs 164.12 μs 0.82 0.05 False
[Dequeue_And_Enqueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Int32%2c%20Int32).Dequeue_And_Enqueue(Size%3a%2010).html>) 462.45 ns 299.31 ns 0.65 0.01 True
[HeapSort - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Int32%2c%20Int32).HeapSort(Size%3a%2010).html>) 178.40 ns 133.31 ns 0.75 0.01 True
[K_Max_Elements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Int32%2c%20Int32).K_Max_Elements(Size%3a%20100).html>) 571.30 ns 332.86 ns 0.58 0.01 False
[K_Max_Elements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Int32%2c%20Int32).K_Max_Elements(Size%3a%201000).html>) 4.31 μs 2.58 μs 0.60 0.01 True
[K_Max_Elements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Int32%2c%20Int32).K_Max_Elements(Size%3a%2010).html>) 122.69 ns 78.16 ns 0.64 0.00 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Dequeue_And_Enqueue(Size: 1000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 164.1187774122807 < 189.76090850846586. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/10/2023 9:01:12 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.482125311064614 (T) = (0 -163484.46807429002) / Math.Sqrt((78672682.49925625 / (299)) + (427468.32614194526 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06961018418710181 = (175716.09802225826 - 163484.46807429002) / 175716.09802225826 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Dequeue_And_Enqueue(Size: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 299.309737353019 < 439.934637423005. IsChangePoint: Marked as a change because one of 3/31/2023 8:29:50 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 162.33264962244186 (T) = (0 -300.5893727522637) / Math.Sqrt((277.537695521463 / (299)) + (0.6942314237913214 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.348560680430033 = (461.42344148138164 - 300.5893727522637) / 461.42344148138164 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.HeapSort(Size: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 133.30674013336812 < 169.20788096004395. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 69.75893434510313 (T) = (0 -134.63849652044297) / Math.Sqrt((7.4557500464756075 / (299)) + (4.9800001066472435 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24865910796581742 = (179.19761581979427 - 134.63849652044297) / 179.19761581979427 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.K_Max_Elements(Size: 100) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 332.85827261068005 < 541.8938222802991. IsChangePoint: Marked as a change because one of 3/31/2023 8:29:50 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 127.85777170821702 (T) = (0 -333.8261985096118) / Math.Sqrt((1364.7513293212935 / (299)) + (0.27153864534065003 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4505924731262268 = (607.6112579111218 - 333.8261985096118) / 607.6112579111218 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.K_Max_Elements(Size: 1000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.5831884262859024 < 4.1033482011752005. IsChangePoint: Marked as a change because one of 3/31/2023 8:29:50 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 258.67344714525984 (T) = (0 -2578.3166935781187) / Math.Sqrt((15317.410628753854 / (299)) + (6.668656231865602 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4191672444445846 = (4439.00015782104 - 2578.3166935781187) / 4439.00015782104 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.K_Max_Elements(Size: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 78.16016693623877 < 116.14597427480078. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 227.15889857294852 (T) = (0 -77.91086306412491) / Math.Sqrt((2.798217177541185 / (299)) + (0.40649384922641363 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3701501782224756 = (123.69752339414742 - 77.91086306412491) / 123.69752339414742 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringD - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString%3a%20123).html>) 28.50 ns 23.64 ns 0.83 0.03 False
[ToByteArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString%3a%20-2147483648).html>) 22.15 ns 15.63 ns 0.71 0.36 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Numerics.Tests.Perf_BigInteger.Parse(numberString%3a%20123).html>) 127.78 ns 85.04 ns 0.67 0.01 True
[ModPow - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Numerics.Tests.Perf_BigInteger.ModPow(arguments%3a%201024%2c1024%2c64%20bits).html>) 123.45 μs 101.60 μs 0.82 0.01 True
[ModPow - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Numerics.Tests.Perf_BigInteger.ModPow(arguments%3a%2016384%2c16384%2c64%20bits).html>) 2.48 ms 2.10 ms 0.85 0.00 True
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Numerics.Tests.Perf_BigInteger.Parse(numberString%3a%20-2147483648).html>) 196.70 ns 141.64 ns 0.72 0.03 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.642448896545897 < 27.023892370259894. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 106.94629335095084 (T) = (0 -23.778464553937262) / Math.Sqrt((1.0078863867697996 / (299)) + (0.030284480018904034 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2534946385518623 = (31.85303921703881 - 23.778464553937262) / 31.85303921703881 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: -2147483648) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.632647479464111 < 19.06346204050109. IsChangePoint: Marked as a change because one of 4/26/2023 8:38:54 PM, 5/11/2023 10:01:37 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.71279659454241 (T) = (0 -15.703129201641106) / Math.Sqrt((1.4250288372548114 / (299)) + (0.10153506151832421 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06486495367136956 = (16.792365191842702 - 15.703129201641106) / 16.792365191842702 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 123) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 85.03540124934806 < 121.45099080822301. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 126.29395707361472 (T) = (0 -84.84709480975492) / Math.Sqrt((11.030190010847672 / (299)) + (0.7950787321077687 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31791362662495287 = (124.39347584371328 - 84.84709480975492) / 124.39347584371328 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 1024,1024,64 bits) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 101.59905206400741 < 117.3047408136483. IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.19419896559553 (T) = (0 -100702.2736280286) / Math.Sqrt((4429154.961690537 / (299)) + (5644562.193503991 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20264578892405433 = (126295.53118198432 - 100702.2736280286) / 126295.53118198432 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 16384,16384,64 bits) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.098893637454982 < 2.351194984914639. IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 193.14300587486244 (T) = (0 -2092888.8080018307) / Math.Sqrt((1166603491.7382886 / (299)) + (15678213.849001365 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17257378126142936 = (2529396.290091563 - 2092888.8080018307) / 2529396.290091563 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: -2147483648) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 141.6416995324379 < 187.48021754407395. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 134.8134744629841 (T) = (0 -143.40892885096943) / Math.Sqrt((26.983735069902092 / (299)) + (0.8265629539192667 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.26938069716591895 = (196.28406790606883 - 143.40892885096943) / 196.28406790606883 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TwoArguments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments.html>) 228.23 ns 199.40 ns 0.87 0.22 False
[FourArguments_DefineMessage - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage.html>) 343.54 ns 314.43 ns 0.92 0.15 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 199.40395073369848 < 212.7264560933014. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.94599803774598 (T) = (0 -201.5860568485806) / Math.Sqrt((92.14437651793392 / (299)) + (9.30485274064984 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14923841150970177 = (236.94776489180836 - 201.5860568485806) / 236.94776489180836 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 314.4264640358578 < 329.2736569490251. IsChangePoint: Marked as a change because one of 3/18/2023 3:39:51 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.938614354283995 (T) = (0 -308.78713558950943) / Math.Sqrt((123.31990158664811 / (299)) + (49.517484033033305 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12738158625130655 = (353.86273166410336 - 308.78713558950943) / 353.86273166410336 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.BubbleSort

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Benchstone.BenchI.BubbleSort.Test.html>) 13.90 μs 13.04 μs 0.94 0.01 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BubbleSort*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.BubbleSort.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.044030341163312 < 13.224783956216847. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 15.499246589171067 (T) = (0 -12965.237771081771) / Math.Sqrt((3762.1676462460046 / (299)) + (57041.71948048292 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07347407151763576 = (13993.38903803657 - 12965.237771081771) / 13993.38903803657 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SinCosPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Double.SinCosPi.html>) 102.46 μs 93.65 μs 0.91 0.00 True
[Hypot - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Double.Hypot.html>) 54.38 μs 47.48 μs 0.87 0.00 False
[CosPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Double.CosPi.html>) 52.99 μs 49.99 μs 0.94 0.00 True
[SinPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Double.SinPi.html>) 53.55 μs 48.74 μs 0.91 0.00 True
[ScaleB - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Double.ScaleB.html>) 14.73 μs 10.08 μs 0.68 0.00 True
[ILogB - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.MathBenchmarks.Double.ILogB.html>) 11.15 μs 6.38 μs 0.57 0.00 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.MathBenchmarks.Double.SinCosPi ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 93.64595808383235 < 97.35992489944697. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 175.5293931567831 (T) = (0 -93500.68274170348) / Math.Sqrt((35166.80023146308 / (299)) + (31864.923416874168 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08688093216778435 = (102397.03236476943 - 93500.68274170348) / 102397.03236476943 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.MathBenchmarks.Double.Hypot ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 47.47971757852076 < 51.45017804616886. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 141.8800126664203 (T) = (0 -47473.66518264765) / Math.Sqrt((566233.7951635373 / (299)) + (535.5272877614327 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11618853793801799 = (53714.69733136173 - 47473.66518264765) / 53714.69733136173 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.MathBenchmarks.Double.CosPi ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 49.98508018653321 < 50.34267641242938. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 143.03898773746482 (T) = (0 -50015.68769249839) / Math.Sqrt((30113.73194680845 / (299)) + (3797.736384059392 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05364330550771348 = (52850.77813004899 - 50015.68769249839) / 52850.77813004899 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.MathBenchmarks.Double.SinPi ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 48.73686234423676 < 50.87734029864553. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 281.9909340837555 (T) = (0 -48755.868035479616) / Math.Sqrt((13100.145909093877 / (299)) + (3015.916242712461 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08763531803059202 = (53439.01292872977 - 48755.868035479616) / 53439.01292872977 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.MathBenchmarks.Double.ScaleB ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.076442618266745 < 13.974672076585806. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 300.4548974167093 (T) = (0 -10088.169268169539) / Math.Sqrt((60356.42229611743 / (299)) + (65.08417232379593 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2998983497970591 = (14409.577902359244 - 10088.169268169539) / 14409.577902359244 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.MathBenchmarks.Double.ILogB ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.3810481049562675 < 10.597159420283443. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 4063.5519779053266 (T) = (0 -6381.138571410094) / Math.Sqrt((30.673043407217676 / (299)) + (16.616351645654593 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4280112723558229 = (11156.056514770471 - 6381.138571410094) / 11156.056514770471 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in GuardedDevirtualization.TwoClassVirtual

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassVirtual.Call(testInput%3a%20pB%20%3d%200.80).html>) 4.11 ns 1.16 ns 0.28 0.01 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassVirtual.Call(testInput%3a%20pB%20%3d%200.20).html>) 3.95 ns 1.23 ns 0.31 0.11 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassVirtual.Call(testInput%3a%20pB%20%3d%200.70).html>) 3.96 ns 1.41 ns 0.36 0.03 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassVirtual.Call(testInput%3a%20pB%20%3d%200.90).html>) 3.50 ns 0.89 ns 0.26 0.02 True
[Call - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/GuardedDevirtualization.TwoClassVirtual.Call(testInput%3a%20pB%20%3d%200.10).html>) 3.61 ns 0.95 ns 0.26 0.05 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.TwoClassVirtual*'
### Payloads [Baseline]() [Compare]() ### Histogram #### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.80) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.158415335589993 < 3.9120812846194895. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 311.86861299004073 (T) = (0 -1.1681048211405765) / Math.Sqrt((0.011770326685877644 / (299)) + (0.0005947939569902013 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7112513616396212 = (4.0454037386063755 - 1.1681048211405765) / 4.0454037386063755 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.20) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2253055391424557 < 3.877028179142699. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 4:09:07 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 3.4366894167796875 (T) = (0 -2.1097002772446674) / Math.Sqrt((0.2195461876370016 / (299)) + (1.4866167735325901 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.47071568299798233 = (3.985948968211399 - 2.1097002772446674) / 3.985948968211399 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.70) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.4062914085747253 < 3.772312091322016. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.34888218736483 (T) = (0 -1.5478013907924093) / Math.Sqrt((0.0185033788984138 / (299)) + (0.2596381855629147 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5992072672148863 = (3.8618499393357704 - 1.5478013907924093) / 3.8618499393357704 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.90) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0.8945551740716999 < 3.320724956150492. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 400.2165541078747 (T) = (0 -0.8934527807017436) / Math.Sqrt((0.011651521232526814 / (299)) + (7.799093277166599E-07 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7367327831841399 = (3.393710738115416 - 0.8934527807017436) / 3.393710738115416 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0.9461461169339861 < 3.430675822162278. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 355.0130794088059 (T) = (0 -0.9467736766657744) / Math.Sqrt((0.015880144582578904 / (299)) + (4.1234603370704727E-07 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.732154985620272 = (3.5347817798972314 - 0.9467736766657744) / 3.5347817798972314 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Formats.Cbor.Tests.Perf_CborWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteCoseKey - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey%3a%20ECDSA_P256).html>) 608.38 ns 486.86 ns 0.80 0.18 False
[WriteCoseKey - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey%3a%20ECDSA_P384).html>) 654.99 ns 521.74 ns 0.80 0.23 False
[WriteCoseKey - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey%3a%20ECDSA_P521).html>) 714.23 ns 587.85 ns 0.82 0.24 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Formats.Cbor.Tests.Perf_CborWriter*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P256) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 486.8597976074776 < 580.874801866615. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.847321604530958 (T) = (0 -514.0822695569469) / Math.Sqrt((235.46324563711687 / (299)) + (431.12999014987207 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16825214962195545 = (618.074659673947 - 514.0822695569469) / 618.074659673947 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P384) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 521.7447040530816 < 621.2720187173057. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.83831430962753 (T) = (0 -543.8027082207038) / Math.Sqrt((1065.2343591351657 / (299)) + (225.4333556013603 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19514505042452965 = (675.6530583647881 - 543.8027082207038) / 675.6530583647881 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P521) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 587.8482584666796 < 675.5322034345863. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.865714668937967 (T) = (0 -607.9731855133047) / Math.Sqrt((2163.582514715392 / (299)) + (1198.2330062404246 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1742286599833294 = (736.2488331225336 - 607.9731855133047) / 736.2488331225336 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20Json4KB).html>) 16.87 μs 13.72 μs 0.81 0.00 True
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20LotsOfStrings).html>) 4.51 μs 3.49 μs 0.77 0.00 True
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20LotsOfNumbers).html>) 7.60 μs 6.38 μs 0.84 0.01 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20Json400B).html>) 2.77 μs 2.29 μs 0.83 0.00 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 6.32 μs 5.20 μs 0.82 0.00 True
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20LotsOfStrings).html>) 5.60 μs 4.73 μs 0.85 0.01 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20BroadTree).html>) 24.30 μs 20.22 μs 0.83 0.01 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20DeepTree).html>) 13.48 μs 11.88 μs 0.88 0.01 True
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20DeepTree).html>) 22.41 μs 19.01 μs 0.85 0.00 True
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20Json4KB).html>) 21.41 μs 17.78 μs 0.83 0.00 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20BroadTree).html>) 31.60 μs 27.12 μs 0.86 0.01 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 377.13 ns 323.29 ns 0.86 0.04 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20Json400B).html>) 2.31 μs 1.92 μs 0.83 0.00 True
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20Json400KB).html>) 1.70 ms 1.40 ms 0.82 0.24 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 409.49 ns 331.02 ns 0.81 0.05 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_ParseThenWrite*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.71972287275566 < 16.121267804842923. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 140.57814978278577 (T) = (0 -13754.787271641144) / Math.Sqrt((17042.036906458827 / (299)) + (5877.13255334316 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.187388028376023 = (16926.636269156454 - 13754.787271641144) / 16926.636269156454 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.488230108815014 < 4.289327939779675. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 111.93495030111353 (T) = (0 -3474.489253945684) / Math.Sqrt((2337.2110342494148 / (299)) + (966.6273639568608 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2260286248554172 = (4489.17022712452 - 3474.489253945684) / 4489.17022712452 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.377001235596679 < 7.244906625792657. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 100.06051099467491 (T) = (0 -6241.8512319097745) / Math.Sqrt((4994.213099888848 / (299)) + (2025.42168221618 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1739274029165697 = (7556.056518455569 - 6241.8512319097745) / 7556.056518455569 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.2878914740709924 < 2.630419795723774. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 90.95012529759215 (T) = (0 -2291.1356122369116) / Math.Sqrt((1128.033827036365 / (299)) + (301.50927106637147 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17090759612402834 = (2763.426129012822 - 2291.1356122369116) / 2763.426129012822 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.2026942788411255 < 6.008103011681969. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 125.1563006539993 (T) = (0 -5157.469201687351) / Math.Sqrt((3403.573349508474 / (299)) + (891.8122601913202 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17832696774472537 = (6276.790157675573 - 5157.469201687351) / 6276.790157675573 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.733272456863412 < 5.3438912201380315. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 102.01511970431919 (T) = (0 -4672.250405743026) / Math.Sqrt((4356.349205417601 / (299)) + (900.0673960610178 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1665851881048088 = (5606.152349414448 - 4672.250405743026) / 5606.152349414448 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: BroadTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.218786471263446 < 22.95184704068229. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 183.3971328599901 (T) = (0 -20111.00926150627) / Math.Sqrt((46533.101843338525 / (299)) + (3517.194188243068 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1584328571804324 = (23897.094169012824 - 20111.00926150627) / 23897.094169012824 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: DeepTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.881659706959706 < 12.800183744765796. IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.270221942377475 (T) = (0 -11701.435987603709) / Math.Sqrt((9460.608880230018 / (299)) + (21905.54127861127 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12479561685210848 = (13369.946738059705 - 11701.435987603709) / 13369.946738059705 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: DeepTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.01020030034248 < 21.34721513913477. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.62444413117161 (T) = (0 -19257.79727470166) / Math.Sqrt((41761.21872936558 / (299)) + (95880.75265381235 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13484580759730516 = (22259.38155742985 - 19257.79727470166) / 22259.38155742985 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json4KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.784311224489798 < 20.407949530417977. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 171.96613429099736 (T) = (0 -17841.50913875416) / Math.Sqrt((31783.34076476004 / (299)) + (3876.1141555833487 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16236885033517323 = (21299.95899256294 - 17841.50913875416) / 21299.95899256294 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.11899945622621 < 29.992665159325522. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 180.27442367143445 (T) = (0 -27008.06401281461) / Math.Sqrt((53049.58197114616 / (299)) + (5598.674718734561 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14133468076145156 = (31453.54005535586 - 27008.06401281461) / 31453.54005535586 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 323.2935937901238 < 360.4414020620071. IsChangePoint: Marked as a change because one of 3/8/2023 2:17:54 AM, 4/12/2023 2:59:13 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.635976029928266 (T) = (0 -319.0699488841625) / Math.Sqrt((114.63601300625454 / (299)) + (24.82444292500685 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15154610197640675 = (376.06044315125536 - 319.0699488841625) / 376.06044315125536 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.9244365416589688 < 2.1974571105999847. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 70.92976136403901 (T) = (0 -1895.7564255165323) / Math.Sqrt((351.4710047270411 / (299)) + (439.8354144809573 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18125307509796743 = (2315.436391707205 - 1895.7564255165323) / 2315.436391707205 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.399026805778491 < 1.620055065371469. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 4/3/2023 7:47:24 AM, 4/18/2023 7:36:21 AM, 5/20/2023 7:49:01 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.116676776023226 (T) = (0 -1401703.7881255113) / Math.Sqrt((25470744440.912083 / (299)) + (35604990.23339154 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.20899227411282575 = (1772048.1636932127 - 1401703.7881255113) / 1772048.1636932127 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 331.0215758930526 < 390.0677638671829. IsChangePoint: Marked as a change because one of 4/21/2023 10:42:26 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.15200067889943 (T) = (0 -333.39598246660773) / Math.Sqrt((80.60755334148593 / (299)) + (25.58813817456007 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18369645495699702 = (408.42157857962553 - 333.39598246660773) / 408.42157857962553 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).LinkedList(Size%3a%20512).html>) 3.79 μs 2.19 μs 0.58 0.16 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).SortedDictionary(Size%3a%20512).html>) 12.52 μs 11.32 μs 0.90 0.00 False
[IEnumerable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).IEnumerable(Size%3a%20512).html>) 3.48 μs 2.62 μs 0.75 0.09 False
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).HashSet(Size%3a%20512).html>) 2.15 μs 1.44 μs 0.67 0.13 False
[ConcurrentQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).ConcurrentQueue(Size%3a%20512).html>) 8.77 μs 7.27 μs 0.83 0.29 False
[ImmutableStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).ImmutableStack(Size%3a%20512).html>) 2.03 μs 696.48 ns 0.34 0.01 True
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).ImmutableList(Size%3a%20512).html>) 23.59 μs 19.95 μs 0.85 0.21 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).Stack(Size%3a%20512).html>) 3.05 μs 2.50 μs 0.82 0.11 False
[ImmutableQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).ImmutableQueue(Size%3a%20512).html>) 4.07 μs 1.84 μs 0.45 0.03 True
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).SortedSet(Size%3a%20512).html>) 10.40 μs 6.97 μs 0.67 0.01 True
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEach(String).Queue(Size%3a%20512).html>) 3.42 μs 2.87 μs 0.84 0.04 False

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<String>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.1911605688046216 < 2.44185919146663. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/25/2023 12:18:49 AM, 3/25/2023 7:34:08 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.080705157447344 (T) = (0 -2218.6665563569845) / Math.Sqrt((144338.8494662203 / (299)) + (1372.3414386761626 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18729517768127438 = (2729.9783333719047 - 2218.6665563569845) / 2729.9783333719047 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<String>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.315925222945287 < 12.000646446824229. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 131.76526674376075 (T) = (0 -11307.82802302479) / Math.Sqrt((35801.688718236044 / (299)) + (316.28161159723203 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12270204314692952 = (12889.381463495896 - 11307.82802302479) / 12889.381463495896 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<String>.IEnumerable(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.621636430925647 < 3.3350312057010223. IsChangePoint: Marked as a change because one of 4/7/2023 2:04:00 PM, 4/8/2023 9:16:19 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.45130427686831 (T) = (0 -2610.734987655164) / Math.Sqrt((181611.88510841536 / (299)) + (65.7104245790907 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2461773879486753 = (3463.3280375481886 - 2610.734987655164) / 3463.3280375481886 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<String>.HashSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.4414728406939101 < 2.0173428854905398. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 93.37084676925888 (T) = (0 -1427.9794917415263) / Math.Sqrt((20534.17804132783 / (299)) + (54.22901702473894 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3581847615219676 = (2224.9074283866544 - 1427.9794917415263) / 2224.9074283866544 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<String>.ConcurrentQueue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.265689643306592 < 8.080121560428449. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 25.027920347619933 (T) = (0 -7308.979050906519) / Math.Sqrt((650052.8815000118 / (299)) + (3821.430440561922 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14538373848775343 = (8552.35195030487 - 7308.979050906519) / 8552.35195030487 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<String>.ImmutableStack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 696.4812995276076 < 1.9675420206057173. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 487.4449818349902 (T) = (0 -700.4149111035089) / Math.Sqrt((532.6980622609493 / (299)) + (79.64964498668661 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6618329688904848 = (2071.2099248867344 - 700.4149111035089) / 2071.2099248867344 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<String>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.946485599322322 < 22.47844343512543. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 76.80503511365598 (T) = (0 -19974.34386285264) / Math.Sqrt((814770.7360444178 / (299)) + (6965.927611642655 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1800415863030895 = (24360.191357504573 - 19974.34386285264) / 24360.191357504573 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.498622112882238 < 2.8986032731862585. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 52.13563380991393 (T) = (0 -2504.7679928742978) / Math.Sqrt((43476.98702114451 / (299)) + (709.860677739322 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22742432544468436 = (3242.100515675709 - 2504.7679928742978) / 3242.100515675709 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<String>.ImmutableQueue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.8351470796651996 < 3.8756358528872896. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 240.59139064030714 (T) = (0 -1810.6785603368103) / Math.Sqrt((27682.395798280853 / (299)) + (91.21001733317344 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5700907049619414 = (4211.768810852336 - 1810.6785603368103) / 4211.768810852336 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<String>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.965169726107227 < 9.890923102756668. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 442.91825650363234 (T) = (0 -6967.464259419548) / Math.Sqrt((13337.6752940109 / (299)) + (216.73083933824867 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.33227775393910125 = (10434.674448129874 - 6967.464259419548) / 10434.674448129874 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEach<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.8736179270268587 < 3.2424701445575788. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.67043014617006 (T) = (0 -2839.5263330600146) / Math.Sqrt((23829.53472133617 / (299)) + (589.6318759309057 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18246009918825293 = (3473.2571832158997 - 2839.5263330600146) / 3473.2571832158997 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.ToStringWithFormat(value%3a%201.7976931348623157E%2b308%2c%20format%3a%20%22G17%22).html>) 194.31 ns 182.34 ns 0.94 0.00 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.ToStringWithFormat(value%3a%20-1.7976931348623157E%2b308%2c%20format%3a%20%22E%22).html>) 143.07 ns 133.23 ns 0.93 0.00 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.ToStringWithFormat(value%3a%201.7976931348623157E%2b308%2c%20format%3a%20%22E%22).html>) 140.93 ns 130.49 ns 0.93 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.Parse(value%3a%20%221.7976931348623157e%2b308%22).html>) 140.29 ns 131.33 ns 0.94 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.Parse(value%3a%20%2212345%22).html>) 64.58 ns 57.36 ns 0.89 0.02 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.TryParse(value%3a%20%2212345%22).html>) 62.03 ns 57.45 ns 0.93 0.01 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.ToStringWithFormat(value%3a%20-1.7976931348623157E%2b308%2c%20format%3a%20%22G17%22).html>) 196.27 ns 183.33 ns 0.93 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.Parse(value%3a%20%22-1.7976931348623157e%2b308%22).html>) 148.19 ns 128.37 ns 0.87 0.01 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22F50%22).html>) 334.50 ns 280.94 ns 0.84 0.09 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22G17%22).html>) 195.57 ns 166.39 ns 0.85 0.00 True
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22E%22).html>) 147.87 ns 133.87 ns 0.91 0.00 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Double.TryParse(value%3a%20%22-1.7976931348623157e%2b308%22).html>) 146.88 ns 124.62 ns 0.85 0.02 False

graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 182.3419784833585 < 184.6106907417783. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.958899029352125 (T) = (0 -182.50651905760682) / Math.Sqrt((11.27730383893259 / (299)) + (5.878333872844402 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06443576676505805 = (195.07641760368065 - 182.50651905760682) / 195.07641760368065 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 133.2296878834565 < 136.01455696895724. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.526249365165068 (T) = (0 -134.37063811718582) / Math.Sqrt((15.475335685587961 / (299)) + (2.6458061831561808 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05850548208963417 = (142.72057410957592 - 134.37063811718582) / 142.72057410957592 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 130.49137321947262 < 134.3210462092503. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.93446424052013 (T) = (0 -131.29172353882953) / Math.Sqrt((16.920499126600486 / (299)) + (0.3929555232890509 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06295070271694833 = (140.1118638256346 - 131.29172353882953) / 140.1118638256346 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 131.3275509067513 < 133.550326788104. IsChangePoint: Marked as a change because one of 2/16/2023 5:39:51 AM, 3/13/2023 2:14:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.019396975133256 (T) = (0 -125.78924240246222) / Math.Sqrt((4.3519005921138385 / (299)) + (32.196991076545366 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.121472516358386 = (143.18190921136465 - 125.78924240246222) / 143.18190921136465 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Double.Parse(value: "12345") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 57.35683457488261 < 60.07022851848265. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 45.423511983353926 (T) = (0 -58.12414298819871) / Math.Sqrt((1.7765503620790104 / (299)) + (0.45776239934221447 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13684247823947726 = (67.3389752424875 - 58.12414298819871) / 67.3389752424875 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Double.TryParse(value: "12345") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 57.4506990192056 < 58.63736659352225. IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 66.7529973404034 (T) = (0 -56.823849119094604) / Math.Sqrt((1.6848281054218808 / (299)) + (0.19603659925966266 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14462264624009016 = (66.43132281830799 - 56.823849119094604) / 66.43132281830799 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 183.32669328207766 < 186.1241195244299. IsChangePoint: Marked as a change because one of 3/6/2023 9:06:14 PM, 3/9/2023 2:25:45 AM, 4/8/2023 3:10:58 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.19417936071886 (T) = (0 -184.12721069272584) / Math.Sqrt((13.309842405419865 / (299)) + (1.8163495950615676 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06980969756222873 = (197.94574315619008 - 184.12721069272584) / 197.94574315619008 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 128.37286655733723 < 140.71157791667358. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.548634423864122 (T) = (0 -126.8090934275367) / Math.Sqrt((17.693331775372098 / (299)) + (11.941458962115362 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1495341453747337 = (149.10544936975927 - 126.8090934275367) / 149.10544936975927 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "F50") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 280.94374187739885 < 318.3398060567587. IsChangePoint: Marked as a change because one of 3/9/2023 3:58:44 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.55507810855057 (T) = (0 -282.8479381470734) / Math.Sqrt((176.30942696842308 / (299)) + (6.789398218516251 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13413952272478974 = (326.66687713610855 - 282.8479381470734) / 326.66687713610855 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G17") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 166.39001245190892 < 185.23151859815545. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.63382429175611 (T) = (0 -167.72161467529315) / Math.Sqrt((7.988062439043105 / (299)) + (4.645085340504488 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15759893652652868 = (199.0994811707939 - 167.72161467529315) / 199.0994811707939 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 133.87028794328262 < 140.2613929267816. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.11009993766974 (T) = (0 -133.7616579026291) / Math.Sqrt((19.354921068492185 / (299)) + (0.6552142513056269 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0860377719903154 = (146.35359515230616 - 133.7616579026291) / 146.35359515230616 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 124.61587559120822 < 139.39548489955993. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 15.062598109131667 (T) = (0 -128.00812346025077) / Math.Sqrt((18.00972634998251 / (299)) + (23.426732538283847 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13836034565244407 = (148.5634079331923 - 128.00812346025077) / 148.5634079331923 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22(%3fi)Tom%7cSawyer%7cHuckleberry%7cFinn%22%2c%20Options%3a%20None).html>) 558.82 ms 440.15 ms 0.79 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22Tom%7cSawyer%7cHuckleberry%7cFinn%22%2c%20Options%3a%20None).html>) 23.53 ms 18.87 ms 0.80 0.03 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22(%5bA-Za-z%5dawyer%7c%5bA-Za-z%5dinn)%5c%5cs%22%2c%20Options%3a%20None).html>) 1.26 secs 1.08 secs 0.85 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22.%7b0%2c2%7d(Tom%7cSawyer%7cHuckleberry%7cFinn)%22%2c%20Options%3a%20None).html>) 7.28 secs 5.46 secs 0.75 0.01 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22Tom.%7b10%2c25%7driver%7criver.%7b10%2c25%7dTom%22%2c%20Options%3a%20None).html>) 79.22 ms 69.24 ms 0.87 0.02 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22.%7b2%2c4%7d(Tom%7cSawyer%7cHuckleberry%7cFinn)%22%2c%20Options%3a%20None).html>) 7.13 secs 5.32 secs 0.75 0.02 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Tom|Sawyer|Huckleberry|Finn", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 440.1525214285714 < 530.8243867445054. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 206.3117221095533 (T) = (0 -439350322.07945913) / Math.Sqrt((13699569807067.545 / (299)) + (3941573593570.4204 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21717125574840737 = (561234274.1700051 - 439350322.07945913) / 561234274.1700051 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom|Sawyer|Huckleberry|Finn", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.872307692307693 < 22.13684345238095. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 131.25229449198096 (T) = (0 -18667597.307340093) / Math.Sqrt((122052563704.74704 / (299)) + (11349006062.354858 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20106589751139603 = (23365628.340550613 - 18667597.307340093) / 23365628.340550613 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "([A-Za-z]awyer|[A-Za-z]inn)\\s", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0776653214285714 < 1.198616255357143. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 106.67134365392555 (T) = (0 -1085123151.7864187) / Math.Sqrt((61341296351214.445 / (299)) + (27788848868930.855 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13078546638176702 = (1248395085.2379727 - 1085123151.7864187) / 1248395085.2379727 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.455224321428572 < 6.913603224782051. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 89.98852024508709 (T) = (0 -5394975249.371654) / Math.Sqrt((5761313151752456 / (299)) + (5227557085352421 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2550678189661262 = (7242236792.460885 - 5394975249.371654) / 7242236792.460885 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom.{10,25}river|river.{10,25}Tom", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 69.23584285714286 < 75.26302105769231. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 106.30314552157253 (T) = (0 -68605048.86799099) / Math.Sqrt((592597386926.0282 / (299)) + (110191800209.60631 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1367848635612651 = (79476188.4633149 - 68605048.86799099) / 79476188.4633149 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.318678986666667 < 6.795111904666667. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 71.67220949420714 (T) = (0 -5410204934.468865) / Math.Sqrt((8819837643705478 / (299)) + (7280715895251627 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.243376345014351 = (7150457032.131095 - 5410204934.468865) / 7150457032.131095 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Hashtable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromString(Mode%3a%20Reflection).html>) 42.11 μs 37.06 μs 0.88 0.10 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromString(Mode%3a%20SourceGen).html>) 43.46 μs 36.86 μs 0.85 0.13 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromStream(Mode%3a%20Reflection).html>) 42.57 μs 39.57 μs 0.93 0.13 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromStream(Mode%3a%20SourceGen).html>) 41.90 μs 36.28 μs 0.87 0.08 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 41.79 μs 36.07 μs 0.86 0.11 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromReader(Mode%3a%20SourceGen).html>) 47.32 μs 44.10 μs 0.93 0.07 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 44.21 μs 35.29 μs 0.80 0.11 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Hashtable&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 37.06223045822102 < 40.24693233018137. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 4/25/2023 1:02:54 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.8913521842758 (T) = (0 -37026.947739760195) / Math.Sqrt((979994.2289447278 / (299)) + (980811.2084556919 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11347326271672767 = (41766.306849613895 - 37026.947739760195) / 41766.306849613895 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 36.863118790681746 < 40.8564153643983. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.384796608235792 (T) = (0 -36884.72548822259) / Math.Sqrt((957975.052773182 / (299)) + (919100.0899083605 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1135787313242931 = (41610.830867503355 - 36884.72548822259) / 41610.830867503355 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 39.56563656280429 < 41.40674780217195. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.700132877113628 (T) = (0 -36913.82522488239) / Math.Sqrt((1294489.73538142 / (299)) + (714500.3927220652 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13519864876104407 = (42684.74508278447 - 36913.82522488239) / 42684.74508278447 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 36.27903823529412 < 40.34996010058769. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.755475915271514 (T) = (0 -36801.06253251128) / Math.Sqrt((784097.1988166585 / (299)) + (824576.7318235514 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12659815703505178 = (42135.315867415906 - 36801.06253251128) / 42135.315867415906 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 36.07375807921777 < 39.375645800075326. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.428488408278756 (T) = (0 -35846.83791535649) / Math.Sqrt((1000325.2584631823 / (299)) + (376812.4568375631 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12862689439204367 = (41138.33406683602 - 35846.83791535649) / 41138.33406683602 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 44.099627677376176 < 45.47283838104768. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.899374113135757 (T) = (0 -42220.43880261348) / Math.Sqrt((746394.4812224794 / (299)) + (951567.4435608377 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10445573921040946 = (47145.00516745898 - 42220.43880261348) / 47145.00516745898 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 35.292324906367035 < 39.920109851313896. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 26.231875821741347 (T) = (0 -35665.86708467346) / Math.Sqrt((756236.3368100267 / (299)) + (465750.5594021815 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12590751377296708 = (40803.310458167885 - 35665.86708467346) / 40803.310458167885 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_DateTimeOffset.TryParse(value%3a%20%2212%2f30%2f2017%203%3a45%3a22%20AM%20-08%3a00%22).html>) 510.85 ns 466.87 ns 0.91 0.00 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20%22r%22).html>) 47.03 ns 41.66 ns 0.89 0.04 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_DateTimeOffset.Parse(value%3a%20%2212%2f30%2f2017%203%3a45%3a22%20AM%20-08%3a00%22).html>) 512.31 ns 464.78 ns 0.91 0.00 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20%22G%22).html>) 214.43 ns 156.85 ns 0.73 0.01 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_DateTimeOffset.TryParse(value: "12/30/2017 3:45:22 AM -08:00") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 466.87316251013794 < 482.7807294313452. IsChangePoint: Marked as a change because one of 3/26/2023 5:20:11 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.02896229095515 (T) = (0 -474.5548963820812) / Math.Sqrt((149.62913721892636 / (299)) + (96.60844237332407 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10623827349357563 = (530.9635468919021 - 474.5548963820812) / 530.9635468919021 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_DateTimeOffset.ToString(format: "r") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 41.65742483978618 < 43.90043323954803. IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 42.568765390603424 (T) = (0 -42.09444969006545) / Math.Sqrt((1.2197958958218647 / (299)) + (0.24817766357397994 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13339832973541615 = (48.57416173362961 - 42.09444969006545) / 48.57416173362961 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_DateTimeOffset.Parse(value: "12/30/2017 3:45:22 AM -08:00") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 464.77877170855794 < 485.9387348045857. IsChangePoint: Marked as a change because one of 3/28/2023 10:24:48 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 36.82408941933327 (T) = (0 -468.5685413196252) / Math.Sqrt((143.89444683521643 / (299)) + (33.349908472377 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12062635010811547 = (532.8435089876002 - 468.5685413196252) / 532.8435089876002 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_DateTimeOffset.ToString(format: "G") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 156.84524491077593 < 203.0908867187902. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 105.64257450464301 (T) = (0 -158.2440310889526) / Math.Sqrt((57.00418078348878 / (299)) + (2.9884150066960378 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3021249140159429 = (226.75122563777504 - 158.2440310889526) / 226.75122563777504 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonFromStream(IndexViewModel).DataContractJsonSerializer.html>) 383.00 μs 305.90 μs 0.80 0.02 True
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_FromStream(IndexViewModel).SystemTextJsonSourceGen.html>) 34.89 μs 27.25 μs 0.78 0.08 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_FromStream(IndexViewModel).SystemTextJsonReflection.html>) 34.58 μs 28.66 μs 0.83 0.12 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonFromStream(IndexViewModel).JsonNet.html>) 82.73 μs 64.11 μs 0.77 0.01 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 305.9017355660212 < 362.7261848903393. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 72.54339510604119 (T) = (0 -308775.832899015) / Math.Sqrt((36157389.039435364 / (299)) + (11761769.343056133 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19220386715909984 = (382244.75253811356 - 308775.832899015) / 382244.75253811356 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.245470832422985 < 32.96094641773771. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 44.336538413590844 (T) = (0 -26962.787755880017) / Math.Sqrt((651083.7667262787 / (299)) + (266351.93608783913 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19843731506833662 = (33637.778133570064 - 26962.787755880017) / 33637.778133570064 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 28.663792602781598 < 32.196489488869716. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 35.21544177012395 (T) = (0 -27968.70948622196) / Math.Sqrt((907456.7893196472 / (299)) + (329488.1434775536 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17499483017090017 = (33901.2536030722 - 27968.70948622196) / 33901.2536030722 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromStream<IndexViewModel>.JsonNet_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 64.11170017273696 < 78.48930171227559. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.23816964472753 (T) = (0 -64916.535661404676) / Math.Sqrt((1512525.51655917 / (299)) + (17425854.064223867 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22489731562006013 = (83752.17499515701 - 64916.535661404676) / 83752.17499515701 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Int32.ToString(value%3a%2012345).html>) 13.33 ns 10.83 ns 0.81 0.17 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Int32.ToString(value%3a%20-2147483648).html>) 27.32 ns 23.10 ns 0.85 0.04 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Int32.ToString(value%3a%204).html>) 3.48 ns 1.31 ns 0.38 0.54 False
[ToStringHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Int32.ToStringHex(value%3a%20-2147483648).html>) 23.81 ns 20.59 ns 0.86 0.30 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int32.ToString(value: 12345) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.832786960420512 < 12.964279394646976. IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.080068944464093 (T) = (0 -11.129703687197244) / Math.Sqrt((0.8273702521883322 / (299)) + (0.28343445414040674 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11338166751068543 = (12.552981682601716 - 11.129703687197244) / 12.552981682601716 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Int32.ToString(value: -2147483648) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.09683704681302 < 25.82962186494085. IsChangePoint: Marked as a change because one of 4/17/2023 2:42:45 PM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 43.20874599314019 (T) = (0 -23.118376759844786) / Math.Sqrt((1.0884601604230237 / (299)) + (0.2036204037393027 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2061450348648576 = (29.121662992822895 - 23.118376759844786) / 29.121662992822895 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Int32.ToString(value: 4) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.3072792281416254 < 3.0455723542701554. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 22.716264176877566 (T) = (0 -1.4040486541245671) / Math.Sqrt((0.35087932832248236 / (299)) + (0.023618235885524374 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4694206824384634 = (2.6462559086874426 - 1.4040486541245671) / 2.6462559086874426 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Int32.ToStringHex(value: -2147483648) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.585796508624814 < 22.93577629937622. IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 4/25/2023 6:46:56 PM, 5/5/2023 8:30:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.52088934514177 (T) = (0 -21.11318897842464) / Math.Sqrt((1.172832517675601 / (299)) + (0.3905517517828374 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10556408806790037 = (23.605032732660927 - 21.11318897842464) / 23.605032732660927 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Int32).SerializeToStream(Mode%3a%20SourceGen).html>) 228.91 ns 179.88 ns 0.79 0.13 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Int32).SerializeToWriter(Mode%3a%20Reflection).html>) 61.07 ns 39.83 ns 0.65 0.07 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Int32).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 140.80 ns 111.60 ns 0.79 0.04 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Int32).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 327.55 ns 261.81 ns 0.80 0.02 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Int32).SerializeToWriter(Mode%3a%20SourceGen).html>) 62.68 ns 41.16 ns 0.66 0.07 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Int32).SerializeToString(Mode%3a%20SourceGen).html>) 176.10 ns 132.69 ns 0.75 0.02 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Int32).SerializeToString(Mode%3a%20Reflection).html>) 170.11 ns 130.91 ns 0.77 0.06 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Int32).SerializeObjectProperty(Mode%3a%20Reflection).html>) 324.56 ns 259.67 ns 0.80 0.01 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Int32).SerializeToStream(Mode%3a%20Reflection).html>) 238.55 ns 177.57 ns 0.74 0.12 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(Int32).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 140.55 ns 110.72 ns 0.79 0.07 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 179.88498600282261 < 219.745651776403. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 33.67716690374289 (T) = (0 -181.10951490844656) / Math.Sqrt((58.467140911666874 / (299)) + (31.912416287740157 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23237708419103956 = (235.93552404253597 - 181.10951490844656) / 235.93552404253597 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 39.82541155710425 < 59.12253841692319. IsChangePoint: Marked as a change because one of 3/20/2023 7:45:43 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 83.72278242405753 (T) = (0 -40.57159890163842) / Math.Sqrt((7.181319420422429 / (299)) + (0.536723389063613 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3452676577754093 = (61.966694304099725 - 40.57159890163842) / 61.966694304099725 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 111.60143117295479 < 133.99761174351832. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/17/2023 8:43:19 PM, 4/24/2023 1:33:55 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 73.62261838586237 (T) = (0 -111.8605999315595) / Math.Sqrt((54.98683085300502 / (299)) + (0.27584222384849816 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22963407127488925 = (145.20450056336105 - 111.8605999315595) / 145.20450056336105 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 261.80795322733525 < 314.9088773090455. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.57650364509843 (T) = (0 -264.8040680235738) / Math.Sqrt((115.80390373097669 / (299)) + (12.287185664075269 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18064479020948518 = (323.18592090392207 - 264.8040680235738) / 323.18592090392207 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 41.161875356681804 < 59.17335031635536. IsChangePoint: Marked as a change because one of 3/13/2023 3:41:35 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 123.89904156575244 (T) = (0 -41.06837812805251) / Math.Sqrt((5.869603150914409 / (299)) + (0.1203056790039722 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3389477938891733 = (62.125771230188306 - 41.06837812805251) / 62.125771230188306 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 132.69017396992075 < 166.701762152376. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 70.06586915791328 (T) = (0 -131.42106861331547) / Math.Sqrt((45.27411593794705 / (299)) + (2.3287672789710983 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23460968992194206 = (171.70464125671066 - 131.42106861331547) / 171.70464125671066 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 130.91016320835945 < 161.84630963173444. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 71.65932004874574 (T) = (0 -131.02713329356368) / Math.Sqrt((72.62415274148763 / (299)) + (0.9689674368097033 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23554947611090135 = (171.40041009713823 - 131.02713329356368) / 171.40041009713823 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 259.6739889587842 < 311.6406928177943. IsChangePoint: Marked as a change because one of 2/3/2023 8:13:11 AM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 65.83915401519283 (T) = (0 -260.75470726689537) / Math.Sqrt((114.3126653680724 / (299)) + (5.62445607147239 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1856285094715592 = (320.1913503844458 - 260.75470726689537) / 320.1913503844458 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 177.57151780559968 < 220.87290277920388. IsChangePoint: Marked as a change because one of 3/13/2023 3:41:35 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 52.45921632552827 (T) = (0 -180.81810617786957) / Math.Sqrt((44.448548204142284 / (299)) + (12.131342169391573 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23180892740026748 = (235.3816812345139 - 180.81810617786957) / 235.3816812345139 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Int32>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 110.71559704177987 < 133.87845964451574. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/12/2023 6:21:10 AM, 4/25/2023 1:02:54 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 54.10743821254608 (T) = (0 -112.2082787904863) / Math.Sqrt((64.1898402519316 / (299)) + (2.142378806301154 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22901848844022962 = (145.53951957094026 - 112.2082787904863) / 145.53951957094026 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CaseInsensitiveNotMatching - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive(Location).CaseInsensitiveNotMatching.html>) 1.49 μs 1.16 μs 0.77 0.02 True
[CaseInsensitiveMatching - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive(Location).CaseInsensitiveMatching.html>) 1.49 μs 1.17 μs 0.78 0.02 True
[MissingProperties - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive(Location).MissingProperties.html>) 988.71 ns 779.43 ns 0.79 0.01 True
[Baseline - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive(Location).Baseline.html>) 1.52 μs 1.18 μs 0.78 0.01 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.CaseInsensitiveNotMatching ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1554435583810443 < 1.4159403549077947. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 68.18852358911671 (T) = (0 -1158.5668327299095) / Math.Sqrt((657.5952601153275 / (299)) + (282.236948904397 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22347704421868916 = (1491.9930236501498 - 1158.5668327299095) / 1491.9930236501498 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.CaseInsensitiveMatching ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1660398156077716 < 1.413583698955906. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 67.13548464697323 (T) = (0 -1164.0186267130587) / Math.Sqrt((669.7879073297661 / (299)) + (284.5338678498478 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2207590249195202 = (1493.785188327448 - 1164.0186267130587) / 1493.785188327448 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.MissingProperties ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 779.4315185648727 < 941.0617540255039. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 59.31371101756359 (T) = (0 -798.9331469231628) / Math.Sqrt((148.01500496275852 / (299)) + (117.62500584255743 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18655882322655773 = (982.164623251768 - 798.9331469231628) / 982.164623251768 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.Baseline ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.180880865399275 < 1.4242739544333918. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 104.6467520850179 (T) = (0 -1148.7624245790753) / Math.Sqrt((549.4561845397578 / (299)) + (115.14267500487324 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22952903641028702 = (1490.9873036965064 - 1148.7624245790753) / 1490.9873036965064 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Cache

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total%3a%20400000%2c%20unique%3a%201%2c%20cacheSize%3a%2015).html>) 46.77 ms 40.67 ms 0.87 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total%3a%20400000%2c%20unique%3a%207%2c%20cacheSize%3a%2015).html>) 58.83 ms 50.88 ms 0.86 0.00 True
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total%3a%2040000%2c%20unique%3a%201600%2c%20cacheSize%3a%20800).html>) 102.79 ms 95.36 ms 0.93 0.43 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Cache*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 400000, unique: 1, cacheSize: 15) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 40.66744888888888 < 44.392968169642856. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 185.98267673454353 (T) = (0 -40601640.619892925) / Math.Sqrt((284629153658.63477 / (299)) + (9535854673.266302 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15828929186512453 = (48237048.93794333 - 40601640.619892925) / 48237048.93794333 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 400000, unique: 7, cacheSize: 15) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 50.8784375 < 56.43715825. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 69.81268745522684 (T) = (0 -49228783.465060584) / Math.Sqrt((516642746956.23627 / (299)) + (287408933414.9942 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17961826255237817 = (60007166.44207801 - 49228783.465060584) / 60007166.44207801 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 40000, unique: 1600, cacheSize: 800) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 95.361195 < 101.507438125. IsChangePoint: Marked as a change because one of 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.217589504622712 (T) = (0 -98004173.33787036) / Math.Sqrt((44740448033675.836 / (299)) + (15425324636434.783 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.11102734039965449 = (110244305.35572375 - 98004173.33787036) / 110244305.35572375 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromReader(Mode%3a%20SourceGen).html>) 215.97 ns 167.09 ns 0.77 0.21 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromStream(Mode%3a%20Reflection).html>) 347.60 ns 285.07 ns 0.82 0.11 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromString(Mode%3a%20Reflection).html>) 145.94 ns 114.34 ns 0.78 0.19 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromString(Mode%3a%20SourceGen).html>) 148.00 ns 114.23 ns 0.77 0.21 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromStream(Mode%3a%20SourceGen).html>) 346.29 ns 280.61 ns 0.81 0.10 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 88.70 ns 75.74 ns 0.85 0.13 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 167.09334467404545 < 196.3819130801767. IsChangePoint: Marked as a change because one of 1/23/2023 5:59:53 PM, 3/2/2023 11:02:42 PM, 3/3/2023 4:59:02 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 26.478057809506936 (T) = (0 -170.54626250711874) / Math.Sqrt((183.9711090757228 / (299)) + (12.56159444209255 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16335379150487156 = (203.84513881187544 - 170.54626250711874) / 203.84513881187544 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 285.06957022888196 < 327.6308450349739. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 35.887014104780015 (T) = (0 -282.4708541899369) / Math.Sqrt((170.34330340222775 / (299)) + (35.28774208552779 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18714855280685277 = (347.50612201692627 - 282.4708541899369) / 347.50612201692627 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 114.33766733427484 < 138.67648760590845. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 91.6645624274613 (T) = (0 -114.20359714435153) / Math.Sqrt((33.808687656292456 / (299)) + (0.6078104715947901 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24293080423070265 = (150.84961557351878 - 114.20359714435153) / 150.84961557351878 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 114.22752071260773 < 140.82124434319834. IsChangePoint: Marked as a change because one of 3/22/2023 9:02:35 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.6844661670605 (T) = (0 -114.79204052376943) / Math.Sqrt((56.805125299213856 / (299)) + (0.6664686688489728 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2446211159708906 = (151.96617611480093 - 114.79204052376943) / 151.96617611480093 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 280.60576308840604 < 327.00058574026014. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 46.916395667358444 (T) = (0 -281.66791969446547) / Math.Sqrt((179.45747655246763 / (299)) + (19.728082286568085 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1951028660851605 = (349.9427539572613 - 281.66791969446547) / 349.9427539572613 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 75.74394678250768 < 84.39961158887303. IsChangePoint: Marked as a change because one of 2/20/2023 2:42:44 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.07120853842386 (T) = (0 -76.39780586236155) / Math.Sqrt((13.91250771465848 / (299)) + (20.38102954753227 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14346279210309248 = (89.19379702131604 - 76.39780586236155) / 89.19379702131604 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int32).GetHashCodeBenchmark.html>) 19.86 ns 18.17 ns 0.91 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.166210526441237 < 19.11116668755349. IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 53.27051231644702 (T) = (0 -18.097944326596476) / Math.Sqrt((0.18445112478687392 / (299)) + (0.012168535160843118 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10393755130627327 = (20.19719089108078 - 18.097944326596476) / 20.19719089108078 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Channels.Tests.BoundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadAsyncThenWriteAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Threading.Channels.Tests.BoundedChannelPerfTests.ReadAsyncThenWriteAsync.html>) 110.75 ns 100.85 ns 0.91 0.04 False
[TryWriteThenTryRead - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Threading.Channels.Tests.BoundedChannelPerfTests.TryWriteThenTryRead.html>) 50.45 ns 43.15 ns 0.86 0.08 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Channels.Tests.BoundedChannelPerfTests.ReadAsyncThenWriteAsync ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 100.85414590842522 < 105.12905139999879. IsChangePoint: Marked as a change because one of 3/27/2023 4:20:55 AM, 3/31/2023 7:02:21 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.60261255730285 (T) = (0 -103.14542275954324) / Math.Sqrt((19.00469191827181 / (299)) + (1.1834740636465495 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12536994154736789 = (117.93034296354377 - 103.14542275954324) / 117.93034296354377 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Threading.Channels.Tests.BoundedChannelPerfTests.TryWriteThenTryRead ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 43.151236254564886 < 44.37877114289927. IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.62642217049276 (T) = (0 -41.40620630965098) / Math.Sqrt((1.9072601253998849 / (299)) + (0.8957574879309974 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14029284124091113 = (48.163151705537935 - 41.40620630965098) / 48.163151705537935 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArrayCopyToIntArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size%3a%204).html>) 11.89 ns 9.41 ns 0.79 0.13 False
[BitArrayCopyToIntArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size%3a%20512).html>) 22.37 ns 19.57 ns 0.88 0.05 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size: 4) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.405210841044035 < 11.059027762390052. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.720752986666575 (T) = (0 -9.334838944616909) / Math.Sqrt((0.09976748330270299 / (299)) + (0.04074526004463214 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19222095213631377 = (11.556178597729827 - 9.334838944616909) / 11.556178597729827 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Tests.Perf_BitArray.BitArrayCopyToIntArray(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.57424297302128 < 21.18833203601173. IsChangePoint: Marked as a change because one of 3/13/2023 3:41:35 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.52346673849313 (T) = (0 -19.818649120087496) / Math.Sqrt((0.2500139249276332 / (299)) + (0.13015065689111943 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1141644464776118 = (22.372830985708013 - 19.818649120087496) / 22.372830985708013 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ArrayList).DeserializeFromReader(Mode%3a%20SourceGen).html>) 28.40 μs 23.62 μs 0.83 0.13 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ArrayList).DeserializeFromString(Mode%3a%20SourceGen).html>) 25.08 μs 20.93 μs 0.83 0.13 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ArrayList).DeserializeFromStream(Mode%3a%20SourceGen).html>) 25.51 μs 24.08 μs 0.94 0.15 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ArrayList).DeserializeFromReader(Mode%3a%20Reflection).html>) 28.48 μs 23.61 μs 0.83 0.17 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ArrayList).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 25.01 μs 20.37 μs 0.81 0.20 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ArrayList).DeserializeFromStream(Mode%3a%20Reflection).html>) 25.74 μs 21.04 μs 0.82 0.15 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ArrayList).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 24.87 μs 20.30 μs 0.82 0.20 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ArrayList&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<ArrayList>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.62217080152672 < 27.685008302009788. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.9111794119609 (T) = (0 -24092.965036067886) / Math.Sqrt((414486.3443724142 / (299)) + (571751.6414925611 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15602328914512031 = (28546.95482256101 - 24092.965036067886) / 28546.95482256101 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ArrayList>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.93487708573827 < 23.86319342353101. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.58839286175639 (T) = (0 -20959.738607142237) / Math.Sqrt((472809.20388531714 / (299)) + (74297.84440677585 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17092863971830824 = (25280.982568280724 - 20959.738607142237) / 25280.982568280724 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ArrayList>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 24.078843576125443 < 24.11821102405554. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.91130611265375 (T) = (0 -21478.167913133228) / Math.Sqrt((532189.4236460122 / (299)) + (724352.3057904249 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1588184511863749 = (25533.33218426549 - 21478.167913133228) / 25533.33218426549 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ArrayList>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.608913690476186 < 27.308674237593674. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.82694137277613 (T) = (0 -23655.708377502007) / Math.Sqrt((647288.9420636455 / (299)) + (238022.34809467723 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.174004717599746 = (28639.03569613745 - 23655.708377502007) / 28639.03569613745 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ArrayList>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.36534598214286 < 23.762894242644034. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.27710688005626 (T) = (0 -20647.925363425646) / Math.Sqrt((628489.5214078955 / (299)) + (127680.89998230721 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16834789715228018 = (24827.599536781785 - 20647.925363425646) / 24827.599536781785 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ArrayList>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 21.03729150644107 < 24.48923847698801. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.223095871214422 (T) = (0 -21830.606607197013) / Math.Sqrt((640476.0156418073 / (299)) + (1628758.7535210077 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14322569587906292 = (25479.996893225612 - 21830.606607197013) / 25479.996893225612 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ArrayList>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.2961385641213 < 23.603017171796317. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.184573548565426 (T) = (0 -21100.250651008082) / Math.Sqrt((526008.4519426202 / (299)) + (1195224.4024069952 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1502112565282088 = (24829.995470172413 - 21100.250651008082) / 24829.995470172413 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.DependencyInjection.TimeToFirstService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Singleton - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode%3a%20%22Dynamic%22).html>) 3.28 μs 2.86 μs 0.87 0.20 False
[Singleton - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode%3a%20%22ILEmit%22).html>) 3.59 μs 2.91 μs 0.81 0.20 False
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22Expressions%22).html>) 3.57 μs 3.10 μs 0.87 0.19 False
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22Runtime%22).html>) 3.48 μs 3.14 μs 0.90 0.15 False
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22Dynamic%22).html>) 3.47 μs 3.15 μs 0.91 0.17 False
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode%3a%20%22ILEmit%22).html>) 3.46 μs 3.18 μs 0.92 0.18 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.TimeToFirstService*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "Dynamic") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.862556155549263 < 3.0679287107701536. IsChangePoint: Marked as a change because one of 2/7/2023 3:56:56 PM, 2/8/2023 4:45:44 PM, 5/3/2023 1:18:01 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.588223778281879 (T) = (0 -2924.6115536804014) / Math.Sqrt((22789.846194889808 / (299)) + (15987.562611234973 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12525805513104538 = (3343.399240010765 - 2924.6115536804014) / 3343.399240010765 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Singleton(Mode: "ILEmit") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.9061662581699346 < 3.0422475685537047. IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.29443815189767 (T) = (0 -2911.911368323543) / Math.Sqrt((22747.230971758967 / (299)) + (5517.756047979264 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12910180033666216 = (3343.5726121023067 - 2911.911368323543) / 3343.5726121023067 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Expressions") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.10141783627555 < 3.306747004291275. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.826651186930082 (T) = (0 -3149.9275995611906) / Math.Sqrt((31774.76886494987 / (299)) + (16590.041026685754 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14894363457965745 = (3701.1973913213374 - 3149.9275995611906) / 3701.1973913213374 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Runtime") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.1435748547445366 < 3.317272676739498. IsChangePoint: Marked as a change because one of 2/15/2023 9:41:15 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.003807057018 (T) = (0 -3105.8761082785527) / Math.Sqrt((28250.487720131823 / (299)) + (599.2057110584664 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15758881295200936 = (3686.8884886990663 - 3105.8761082785527) / 3686.8884886990663 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "Dynamic") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.1489397171691036 < 3.31264869322918. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.442017551392823 (T) = (0 -3152.078735164102) / Math.Sqrt((27599.51470573296 / (299)) + (16760.185815153443 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14551814151577552 = (3688.877304844847 - 3152.078735164102) / 3688.877304844847 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.DependencyInjection.TimeToFirstService.Scoped(Mode: "ILEmit") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.180739721878707 < 3.313017104682497. IsChangePoint: Marked as a change because one of 4/27/2023 10:53:57 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.301240573519415 (T) = (0 -3100.875245265949) / Math.Sqrt((32711.724246331785 / (299)) + (1947.8425110147725 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16225559487932026 = (3701.45742103674 - 3100.875245265949) / 3701.45742103674 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20).html>) 182.33 ns 119.64 ns 0.66 0.03 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20ja).html>) 181.88 ns 120.10 ns 0.66 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring%3a%20fr).html>) 408.26 ns 350.47 ns 0.86 0.01 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20fr).html>) 181.63 ns 118.44 ns 0.65 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring%3a%20da).html>) 426.19 ns 371.86 ns 0.87 0.01 True
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20da).html>) 159.94 ns 119.23 ns 0.75 0.04 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring%3a%20).html>) 405.22 ns 352.64 ns 0.87 0.01 False
[ToStringHebrewIsrael - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael.html>) 353.72 ns 286.80 ns 0.81 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring%3a%20ja).html>) 404.83 ns 351.15 ns 0.87 0.00 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 119.64346717210469 < 171.35956646577455. IsChangePoint: Marked as a change because one of 4/24/2023 1:33:55 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 100.89153090949974 (T) = (0 -119.06698587087642) / Math.Sqrt((199.39227601647923 / (299)) + (0.4193865908956454 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41469163993208236 = (203.42608100977782 - 119.06698587087642) / 203.42608100977782 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ja) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 120.09733953093252 < 171.6820753371118. IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 95.16116330147558 (T) = (0 -117.70741441791353) / Math.Sqrt((177.9852405479009 / (299)) + (1.7081216031066058 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.40799098491221486 = (198.82706414607463 - 117.70741441791353) / 198.82706414607463 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: fr) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 350.4691155797764 < 387.84646418470174. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.76511866073633 (T) = (0 -355.6495584005372) / Math.Sqrt((30.97035108619936 / (299)) + (62.28844958492704 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10949399616548934 = (399.3791809028951 - 355.6495584005372) / 399.3791809028951 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: fr) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 118.43621156207814 < 170.71048676474481. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 96.42110312357433 (T) = (0 -119.19101987222561) / Math.Sqrt((217.6722966667145 / (299)) + (0.15779293661108706 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.41036395267142634 = (202.14337371712048 - 119.19101987222561) / 202.14337371712048 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: da) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 371.8634183475371 < 405.7331776827211. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 25.739390819277215 (T) = (0 -374.8947230345119) / Math.Sqrt((19.197642523704953 / (299)) + (46.5244560672594 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11586157132342707 = (424.0226540041642 - 374.8947230345119) / 424.0226540041642 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: da) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 119.23386545384739 < 153.06875100667074. IsChangePoint: Marked as a change because one of 4/19/2023 6:20:25 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.42897357602797 (T) = (0 -118.9394823966083) / Math.Sqrt((267.4638949782209 / (299)) + (4.0268798147121405 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3728271919102664 = (189.64387623704323 - 118.9394823966083) / 189.64387623704323 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: ) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 352.6435559939233 < 384.9426362900108. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 89.94958547942535 (T) = (0 -351.80456057258067) / Math.Sqrt((37.501288633599074 / (299)) + (1.80701201047902 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11619891835288708 = (398.0585313574558 - 351.80456057258067) / 398.0585313574558 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 286.8048241533308 < 332.8815282930544. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 70.70196096006951 (T) = (0 -288.63184332582426) / Math.Sqrt((206.85025603927875 / (299)) + (4.940653423447476 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20229976897928165 = (361.8299608068281 - 288.63184332582426) / 361.8299608068281 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: ja) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 351.14542600014624 < 385.0572133739685. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 74.73816373025745 (T) = (0 -353.93145687274006) / Math.Sqrt((25.83321513577463 / (299)) + (3.7263233928408606 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11423958831590797 = (399.57922278306825 - 353.93145687274006) / 399.57922278306825 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ForEach - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount%3a%200%2c%20hostPortCount%3a%200).html>) 33.05 ns 27.20 ns 0.82 0.53 False
[GetCredential_Uri - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString%3a%20%22http%3a%2f%2fnotfound%22%2c%20uriCount%3a%200).html>) 148.35 ns 115.60 ns 0.78 0.04 True
[ForEach - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount%3a%2010%2c%20hostPortCount%3a%2010).html>) 401.85 ns 293.58 ns 0.73 0.15 False
[GetCredential_HostPort - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host%3a%20%22name5%22%2c%20hostPortCount%3a%2010).html>) 73.31 ns 65.99 ns 0.90 0.17 False
[GetCredential_Uri - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString%3a%20%22http%3a%2f%2fname5%22%2c%20uriCount%3a%2010).html>) 869.71 ns 485.03 ns 0.56 0.01 True
[ForEach - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount%3a%200%2c%20hostPortCount%3a%2010).html>) 248.25 ns 156.69 ns 0.63 0.22 False
[GetCredential_Uri - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString%3a%20%22http%3a%2f%2fnotfound%22%2c%20uriCount%3a%2010).html>) 694.27 ns 375.87 ns 0.54 0.01 True
[ForEach - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount%3a%2010%2c%20hostPortCount%3a%200).html>) 260.68 ns 160.22 ns 0.61 0.14 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 0) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.195288797801588 < 32.7081989546934. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.83067159010772 (T) = (0 -28.247591710280954) / Math.Sqrt((3.558531383373726 / (299)) + (1.6402117537226741 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1985291404160836 = (35.2446896509072 - 28.247591710280954) / 35.2446896509072 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 0) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 115.60139595192639 < 140.82207374868895. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.75619970747908 (T) = (0 -118.316476124201) / Math.Sqrt((7.363577639301837 / (299)) + (7.628242918705234 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21627549130017626 = (150.9669211704054 - 118.316476124201) / 150.9669211704054 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 293.5779434016498 < 380.30923120773775. IsChangePoint: Marked as a change because one of 1/30/2023 1:22:46 AM, 1/30/2023 9:42:46 PM, 3/5/2023 5:11:19 AM, 3/5/2023 11:54:18 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 40.53812769947601 (T) = (0 -294.21010019628255) / Math.Sqrt((2494.1927771925957 / (299)) + (2.715992837516223 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28719543248142276 = (412.75002097768515 - 294.21010019628255) / 412.75002097768515 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_HostPort(host: "name5", hostPortCount: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 65.992756952752 < 68.8151169896471. IsChangePoint: Marked as a change because one of 4/5/2023 11:58:06 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.403562058742025 (T) = (0 -64.92847286448003) / Math.Sqrt((5.482824391861419 / (299)) + (6.220378307027473 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15115849174008966 = (76.49069023212671 - 64.92847286448003) / 76.49069023212671 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://name5", uriCount: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 485.0254344923272 < 827.6593968366542. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 307.3999143807979 (T) = (0 -485.15758717074766) / Math.Sqrt((94.10299630431105 / (299)) + (16.890560598828305 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4459701519968778 = (875.6885372139978 - 485.15758717074766) / 875.6885372139978 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 156.68536932319714 < 203.5815044961753. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 35.353903012099885 (T) = (0 -155.55479488450482) / Math.Sqrt((1144.229780439794 / (299)) + (1.9293294400790861 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3118373475279749 = (226.04364582372966 - 155.55479488450482) / 226.04364582372966 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Primitives.Tests.CredentialCacheTests.GetCredential_Uri(uriString: "http://notfound", uriCount: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 375.87057141472405 < 658.3622891688515. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 178.65839082352528 (T) = (0 -377.8033744800257) / Math.Sqrt((19.48087848472162 / (299)) + (40.57047060755095 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.45772078067703204 = (696.6952835694326 - 377.8033744800257) / 696.6952835694326 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 0) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 160.21676578000412 < 224.50020901139013. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 53.434868649818625 (T) = (0 -157.99065174917178) / Math.Sqrt((740.7612046898552 / (299)) + (2.930695189030969 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35734423374206575 = (245.84024612293163 - 157.99065174917178) / 245.84024612293163 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Net.Http.Tests.SocketsHttpHandlerPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Get_EnumerateHeaders_Unvalidated - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Unvalidated(ssl%3a%20False%2c%20chunkedResponse%3a%20False%2c%20responseLength%3a%201).html>) 97.82 μs 92.92 μs 0.95 0.08 False
[Get_EnumerateHeaders_Validated - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Validated(ssl%3a%20False%2c%20chunkedResponse%3a%20False%2c%20responseLength%3a%20100000).html>) 124.60 μs 116.86 μs 0.94 0.06 False
[Get_EnumerateHeaders_Validated - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Validated(ssl%3a%20False%2c%20chunkedResponse%3a%20True%2c%20responseLength%3a%20100000).html>) 136.22 μs 120.34 μs 0.88 0.12 False
[Get_EnumerateHeaders_Unvalidated - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Unvalidated(ssl%3a%20False%2c%20chunkedResponse%3a%20False%2c%20responseLength%3a%20100000).html>) 126.56 μs 116.50 μs 0.92 0.05 False
[Get - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get(ssl%3a%20False%2c%20chunkedResponse%3a%20False%2c%20responseLength%3a%20100000).html>) 125.33 μs 116.51 μs 0.93 0.03 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Http.Tests.SocketsHttpHandlerPerfTest*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Unvalidated(ssl: False, chunkedResponse: False, responseLength: 1) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 92.92173815740395 < 93.19268059448804. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.352823210730556 (T) = (0 -91999.97925551311) / Math.Sqrt((3357385.431180289 / (299)) + (5158324.692586779 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0730647377153667 = (99251.78488598997 - 91999.97925551311) / 99251.78488598997 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Validated(ssl: False, chunkedResponse: False, responseLength: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 116.86406292195518 < 118.39649328342742. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.78315525233196 (T) = (0 -117453.31025608705) / Math.Sqrt((7130818.380148478 / (299)) + (1916694.6256930411 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06207725156322835 = (125227.06209210251 - 117453.31025608705) / 125227.06209210251 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Validated(ssl: False, chunkedResponse: True, responseLength: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 120.33918290328242 < 129.41650499690746. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.341014447394482 (T) = (0 -123770.26067754022) / Math.Sqrt((15063386.5847749 / (299)) + (13057722.00301308 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09966976465552634 = (137472.06949035174 - 123770.26067754022) / 137472.06949035174 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Unvalidated(ssl: False, chunkedResponse: False, responseLength: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 116.50449799196787 < 118.0931068596665. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.882239219753465 (T) = (0 -117397.90999951649) / Math.Sqrt((7215506.6045011645 / (299)) + (1422329.1391937889 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06103449814020621 = (125028.99176486074 - 117397.90999951649) / 125028.99176486074 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get(ssl: False, chunkedResponse: False, responseLength: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 116.50882874475037 < 117.30672942276419. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.97911773508295 (T) = (0 -116184.41378495496) / Math.Sqrt((2334041.676182161 / (299)) + (1164714.9786444155 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06287871474218179 = (123980.12467830205 - 116184.41378495496) / 123980.12467830205 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<StructRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(StructRecord).DeserializeFromString(Mode%3a%20Reflection).html>) 874.86 ns 651.29 ns 0.74 0.16 True
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(StructRecord).DeserializeFromReader(Mode%3a%20Reflection).html>) 1.29 μs 983.68 ns 0.76 0.16 True
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(StructRecord).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 793.00 ns 584.03 ns 0.74 0.04 True
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(StructRecord).DeserializeFromStream(Mode%3a%20Reflection).html>) 1.21 μs 844.34 ns 0.70 0.13 True
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(StructRecord).DeserializeFromString(Mode%3a%20SourceGen).html>) 892.00 ns 637.82 ns 0.72 0.15 True
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(StructRecord).DeserializeFromReader(Mode%3a%20SourceGen).html>) 1.27 μs 921.58 ns 0.73 0.14 True
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(StructRecord).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 793.60 ns 558.32 ns 0.70 0.03 True
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(StructRecord).DeserializeFromStream(Mode%3a%20SourceGen).html>) 1.19 μs 838.02 ns 0.70 0.10 True

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;StructRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 651.2878848740196 < 830.6798479009556. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.54704653566252 (T) = (0 -645.0572827322669) / Math.Sqrt((632.9935329475089 / (299)) + (208.86832117249674 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2720988565073326 = (886.1880332226253 - 645.0572827322669) / 886.1880332226253 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 983.6799409850693 < 1.1890292468447305. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.45837481806462 (T) = (0 -960.1302089400668) / Math.Sqrt((1173.4461488816673 / (299)) + (550.1813358742734 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2515756967747624 = (1282.8688282869891 - 960.1302089400668) / 1282.8688282869891 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 584.0323545222416 < 752.0796770860865. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.09791217671815 (T) = (0 -576.7310824017575) / Math.Sqrt((214.65625820999364 / (299)) + (100.64297476011023 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27469565171610866 = (795.1573484514934 - 576.7310824017575) / 795.1573484514934 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 844.3390503757371 < 1.143989661517931. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 70.35999556581075 (T) = (0 -863.0298590822105) / Math.Sqrt((1371.1089458085366 / (299)) + (213.63429922455293 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27207709114180645 = (1185.606124741895 - 863.0298590822105) / 1185.606124741895 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 637.8170871559633 < 880.6967695028294. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 52.55870487662696 (T) = (0 -661.0959707033005) / Math.Sqrt((769.9289816130238 / (299)) + (248.3351758421752 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27015707245416876 = (905.8058189675152 - 661.0959707033005) / 905.8058189675152 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 921.5785457847431 < 1.2125553594016056. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.75400678245127 (T) = (0 -954.5347296119534) / Math.Sqrt((974.3109773789988 / (299)) + (948.725087305058 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2617169560170494 = (1292.9116243308938 - 954.5347296119534) / 1292.9116243308938 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 558.3151755673276 < 760.8123392403168. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 79.5538395703207 (T) = (0 -573.7143445276354) / Math.Sqrt((241.56755800791706 / (299)) + (99.21919905041916 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.28716424429305704 = (804.8338483788089 - 573.7143445276354) / 804.8338483788089 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<StructRecord>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 838.0172084473746 < 1.1406152582901563. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 68.35299928036314 (T) = (0 -869.7575663772415) / Math.Sqrt((959.6948750402469 / (299)) + (246.63263558155165 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2701421473910988 = (1191.6807680677327 - 869.7575663772415) / 1191.6807680677327 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.Perf_FileStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SeekForward - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.Perf_FileStream.SeekForward(fileSize%3a%201024%2c%20options%3a%20Asynchronous).html>) 41.69 μs 27.96 μs 0.67 0.13 True
[ReadAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.Perf_FileStream.ReadAsync(fileSize%3a%20104857600%2c%20userBufferSize%3a%204096%2c%20options%3a%20None).html>) 115.61 ms 105.23 ms 0.91 0.44 False
[SeekBackward - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.Perf_FileStream.SeekBackward(fileSize%3a%201024%2c%20options%3a%20Asynchronous).html>) 45.31 μs 34.09 μs 0.75 0.21 True
[SeekBackward - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.Perf_FileStream.SeekBackward(fileSize%3a%201024%2c%20options%3a%20None).html>) 43.84 μs 30.45 μs 0.69 0.23 False
[SeekForward - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.Perf_FileStream.SeekForward(fileSize%3a%201024%2c%20options%3a%20None).html>) 41.10 μs 27.09 μs 0.66 0.19 False
[ReadAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.Perf_FileStream.ReadAsync(fileSize%3a%201048576%2c%20userBufferSize%3a%204096%2c%20options%3a%20None).html>) 1.86 ms 1.10 ms 0.59 0.75 False
[GetLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.Perf_FileStream.GetLength(options%3a%20Read).html>) 50.10 ns 28.79 ns 0.57 0.27 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileStream*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_FileStream.SeekForward(fileSize: 1024, options: Asynchronous) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.961448217480193 < 39.72914359195403. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.704957702704256 (T) = (0 -28042.08118977243) / Math.Sqrt((1442437.2966432231 / (299)) + (952683.7166373563 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3267855855797941 = (41654.01184097221 - 28042.08118977243) / 41654.01184097221 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.Perf_FileStream.ReadAsync(fileSize: 104857600, userBufferSize: 4096, options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 105.22568666666668 < 111.019778125. IsChangePoint: Marked as a change because one of 5/17/2023 10:06:24 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.882742629490728 (T) = (0 -107049344.22571914) / Math.Sqrt((58561530224150.31 / (174)) + (13328090441255.115 / (18))) is greater than 1.9725281820011344 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (174) + (18) - 2, .975) and 0.11102567057588923 = (120418937.51315306 - 107049344.22571914) / 120418937.51315306 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.Perf_FileStream.SeekBackward(fileSize: 1024, options: Asynchronous) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 34.088120686753065 < 45.28170640118001. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.302787333329135 (T) = (0 -33121.183790418334) / Math.Sqrt((2490967.6770252655 / (299)) + (2253711.2340008058 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3062598003131578 = (47742.92134918721 - 33121.183790418334) / 47742.92134918721 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.Perf_FileStream.SeekBackward(fileSize: 1024, options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 30.447273565859096 < 41.54588036627594. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.21102747705563 (T) = (0 -30072.07133888601) / Math.Sqrt((6619634.976267226 / (180)) + (1565798.266017082 / (13))) is greater than 1.97246198976699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (180) + (13) - 2, .975) and 0.2711229029857567 = (41258.082414816716 - 30072.07133888601) / 41258.082414816716 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.Perf_FileStream.SeekForward(fileSize: 1024, options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.093876527050607 < 38.86364040133996. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.299740204282347 (T) = (0 -27428.953396915367) / Math.Sqrt((5658399.764806762 / (179)) + (990975.6250019199 / (13))) is greater than 1.9725281820011344 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (179) + (13) - 2, .975) and 0.27258585276505654 = (37707.478609233374 - 27428.953396915367) / 37707.478609233374 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.Perf_FileStream.ReadAsync(fileSize: 1048576, userBufferSize: 4096, options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1042106578947368 < 2.1268986679687503. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.226195055831074 (T) = (0 -825925.6473581106) / Math.Sqrt((416881092586.8308 / (180)) + (8584168516.731505 / (13))) is greater than 1.97246198976699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (180) + (13) - 2, .975) and 0.6154198083775951 = (2147603.192649701 - 825925.6473581106) / 2147603.192649701 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.Perf_FileStream.GetLength(options: Read) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 28.790032685692626 < 47.71513255884716. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 15.72823041584208 (T) = (0 -28.713482044898015) / Math.Sqrt((56.41699652804375 / (299)) + (0.7992338372513369 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21505386258690698 = (36.58019407487955 - 28.713482044898015) / 36.58019407487955 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Ackermann

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Benchstone.BenchI.Ackermann.Test.html>) 4.17 μs 3.18 μs 0.76 0.00 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Ackermann*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.Ackermann.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.180845534805988 < 3.9621354109949016. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 939.8815849698049 (T) = (0 -3179.0923220367686) / Math.Sqrt((68.91948655322567 / (299)) + (11.448443664547119 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23759695223240293 = (4169.831601992559 - 3179.0923220367686) / 4169.831601992559 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchF.Secant

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Benchstone.BenchF.Secant.Test.html>) 220.50 ms 159.80 ms 0.72 0.00 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Secant*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.Secant.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 159.7984857142857 < 209.2600844230769. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 1453.5398196369047 (T) = (0 -159696432.35066214) / Math.Sqrt((89365133230.77628 / (299)) + (18680958968.69655 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27495248472110423 = (220256505.93290225 - 159696432.35066214) / 220256505.93290225 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToStream(Mode%3a%20Reflection).html>) 6.76 μs 5.29 μs 0.78 0.12 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeObjectProperty(Mode%3a%20Reflection).html>) 7.88 μs 7.12 μs 0.90 0.13 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToWriter(Mode%3a%20Reflection).html>) 6.56 μs 5.03 μs 0.77 0.10 True
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 5.07 μs 4.40 μs 0.87 0.00 True
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToStream(Mode%3a%20SourceGen).html>) 4.89 μs 4.18 μs 0.85 0.00 True
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 5.87 μs 5.15 μs 0.88 0.01 True
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 6.90 μs 5.40 μs 0.78 0.09 True
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToString(Mode%3a%20SourceGen).html>) 5.69 μs 4.94 μs 0.87 0.00 True
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToString(Mode%3a%20Reflection).html>) 7.46 μs 6.04 μs 0.81 0.13 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToWriter(Mode%3a%20SourceGen).html>) 4.90 μs 4.09 μs 0.84 0.01 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.292850020609058 < 6.435944675804832. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 53.92222858940001 (T) = (0 -5378.781374472033) / Math.Sqrt((35216.855313167755 / (299)) + (9646.532485623646 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22717868573897362 = (6959.928867406093 - 5378.781374472033) / 6959.928867406093 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.123482457901832 < 7.478963213785137. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.535406659292452 (T) = (0 -6326.398933736419) / Math.Sqrt((46517.920507053685 / (299)) + (60921.21219890757 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20563227712908608 = (7964.06846802922 - 6326.398933736419) / 7964.06846802922 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.0341237113402055 < 6.28214157091516. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 82.47539972419229 (T) = (0 -5075.9929789854805) / Math.Sqrt((31438.071941287744 / (299)) + (4484.413363084446 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2563467806703421 = (6825.75271248213 - 5075.9929789854805) / 6825.75271248213 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.395355464023085 < 4.800766870869111. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 144.25403622627854 (T) = (0 -4369.354550172324) / Math.Sqrt((7986.346989628959 / (299)) + (147.11014321329333 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1691511845681375 = (5258.90447096708 - 4369.354550172324) / 5258.90447096708 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.17786477819348 < 4.670068403771859. IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 131.13002501121755 (T) = (0 -4214.213662317184) / Math.Sqrt((8365.06563223629 / (299)) + (242.36123345023327 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1752286785793916 = (5109.554070161543 - 4214.213662317184) / 5109.554070161543 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.154185823754789 < 5.589989437581135. IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 114.52722369426472 (T) = (0 -5144.707116100841) / Math.Sqrt((9632.538845769295 / (299)) + (482.4553572987438 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15637011058976397 = (6098.298769022275 - 5144.707116100841) / 6098.298769022275 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.402077705214632 < 6.639791526906273. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 137.5353249499129 (T) = (0 -5427.960897315663) / Math.Sqrt((36940.31498856242 / (299)) + (359.5314115683364 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2375555547457762 = (7119.155934706566 - 5427.960897315663) / 7119.155934706566 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.93974853592909 < 5.365458064339791. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 142.88694854304092 (T) = (0 -4944.829951140434) / Math.Sqrt((7080.009903621943 / (299)) + (190.2518925595905 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15172437901265953 = (5829.272737303186 - 4944.829951140434) / 5829.272737303186 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.036012310711501 < 7.0685610599968705. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 78.6216033995438 (T) = (0 -5997.395837945915) / Math.Sqrt((49298.34517197178 / (299)) + (3718.0797509037807 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2177495226460746 = (7666.84842204631 - 5997.395837945915) / 7666.84842204631 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.092033690946733 < 4.645545748349832. IsChangePoint: Marked as a change because one of 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.19471811692569 (T) = (0 -4293.1601067845795) / Math.Sqrt((3144.6544580286095 / (299)) + (13981.675611722756 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13992849960106485 = (4991.631631548356 - 4293.1601067845795) / 4991.631631548356 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.DelegatePerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DelegateInvoke - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.DelegatePerf.DelegateInvoke.html>) 382.18 μs 127.44 μs 0.33 0.24 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.DelegatePerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.DelegatePerf.DelegateInvoke ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 127.44263211382112 < 363.14346887703255. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 2/25/2023 3:43:07 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 44.929434515954426 (T) = (0 -127480.73789393323) / Math.Sqrt((9100817544.573845 / (299)) + (6997.193418017113 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6603769940327123 = (375359.5476574167 - 127480.73789393323) / 375359.5476574167 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).ImmutableSortedSet(Size%3a%20512).html>) 207.42 μs 137.00 μs 0.66 0.12 True
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).ImmutableList(Size%3a%20512).html>) 115.60 μs 107.96 μs 0.93 0.12 False
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 188.21 μs 164.60 μs 0.87 0.32 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).SortedDictionary(Size%3a%20512).html>) 62.14 μs 46.79 μs 0.75 0.11 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).SortedList(Size%3a%20512).html>) 71.67 μs 64.80 μs 0.90 0.00 True
[ICollection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).ICollection(Size%3a%20512).html>) 2.12 μs 1.89 μs 0.89 0.13 False
[IDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).IDictionary(Size%3a%20512).html>) 13.88 μs 10.49 μs 0.76 0.02 True
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).SortedSet(Size%3a%20512).html>) 50.99 μs 39.77 μs 0.78 0.21 False
[ImmutableHashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).ImmutableHashSet(Size%3a%20512).html>) 235.38 μs 179.04 μs 0.76 0.21 False
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CreateAddAndClear(Int32).ImmutableDictionary(Size%3a%20512).html>) 270.68 μs 206.20 μs 0.76 0.12 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 136.99787027665317 < 173.6449010160169. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.68563144031529 (T) = (0 -139263.85582178467) / Math.Sqrt((27488311.57543158 / (299)) + (5455772.880177508 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24937870745425117 = (185531.44868761743 - 139263.85582178467) / 185531.44868761743 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndClear<Int32>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 107.96380028735632 < 110.08714487179488. IsChangePoint: Marked as a change because one of 5/16/2023 5:40:18 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.196585675643174 (T) = (0 -109335.81640092001) / Math.Sqrt((15591384.249543056 / (299)) + (2596744.891410991 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.06516229335897974 = (116957.0029366661 - 109335.81640092001) / 116957.0029366661 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndClear<Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 164.60435267857144 < 183.42856169036372. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 28.99310698428964 (T) = (0 -146048.22332403663) / Math.Sqrt((55101197.90578378 / (299)) + (34946506.71711686 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2517517628223623 = (195186.85920988556 - 146048.22332403663) / 195186.85920988556 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndClear<Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 46.79083094175171 < 58.488785060453864. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 116.24571326953739 (T) = (0 -46862.588088597404) / Math.Sqrt((3282548.118204115 / (299)) + (47957.040958568155 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2310171690678761 = (60941.0070076504 - 46862.588088597404) / 60941.0070076504 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndClear<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 64.79897247638726 < 67.81380614828474. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 98.44338353240165 (T) = (0 -64662.99441716798) / Math.Sqrt((311603.38335061964 / (299)) + (48730.220886323405 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09532754656430033 = (71476.69211281446 - 64662.99441716798) / 71476.69211281446 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndClear<Int32>.ICollection(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.889013940064395 < 2.0111708322873305. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 83.8389174395117 (T) = (0 -1885.0936633707465) / Math.Sqrt((2843.1682767105235 / (299)) + (12.688800414058045 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12588277936050254 = (2156.568500036673 - 1885.0936633707465) / 2156.568500036673 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndClear<Int32>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.49252854264607 < 13.143284886836465. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 45.99253222827839 (T) = (0 -10574.129962157178) / Math.Sqrt((20872.05164345066 / (299)) + (73057.75875197879 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24703548628004945 = (14043.331085972008 - 10574.129962157178) / 14043.331085972008 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndClear<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 39.7713531037415 < 45.95846756584578. IsChangePoint: Marked as a change because one of 3/25/2023 7:34:08 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.79171646335209 (T) = (0 -39971.023094800235) / Math.Sqrt((4104597.425973664 / (299)) + (274101.66668322653 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1566476968824026 = (47395.40396942114 - 39971.023094800235) / 47395.40396942114 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndClear<Int32>.ImmutableHashSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 179.04289325842694 < 225.98108292748915. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.53040800389692 (T) = (0 -180064.26361918807) / Math.Sqrt((77591292.84293379 / (299)) + (18919998.352575216 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2687321051706555 = (246235.7022541097 - 180064.26361918807) / 246235.7022541097 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CreateAddAndClear<Int32>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 206.19615555555558 < 253.32647364403735. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 54.76934461035549 (T) = (0 -205960.46472343788) / Math.Sqrt((61377371.376777045 / (299)) + (13151375.713126654 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22681350704650124 = (266378.7670897981 - 205960.46472343788) / 266378.7670897981 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_ToStream(Location).SystemTextJsonReflection.html>) 871.46 ns 757.97 ns 0.87 0.00 True
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonToStream(Location).DataContractJsonSerializer.html>) 2.80 μs 2.06 μs 0.74 0.01 True
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonToStream(Location).JsonNet.html>) 1.91 μs 1.14 μs 0.60 0.04 True
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_ToStream(Location).SystemTextJsonSourceGen.html>) 630.98 ns 560.35 ns 0.89 0.00 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<Location>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 757.974878948309 < 818.4231608199763. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.86342208882308 (T) = (0 -745.2938092450541) / Math.Sqrt((129.32287147846105 / (299)) + (106.62020984284237 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14166760834353342 = (868.3044196977547 - 745.2938092450541) / 868.3044196977547 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToStream<Location>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.0595381656853364 < 2.6675152081118014. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 85.78953941409175 (T) = (0 -2028.5625908539741) / Math.Sqrt((10894.591064025533 / (299)) + (953.8089947566555 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3070760183140985 = (2927.539880952641 - 2028.5625908539741) / 2927.539880952641 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToStream<Location>.JsonNet_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1395086536056296 < 1.7976570460090417. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 89.79410018787041 (T) = (0 -1175.684572339725) / Math.Sqrt((798.7895139047814 / (299)) + (720.2521726544998 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.36790516841530285 = (1859.9813091213196 - 1175.684572339725) / 1859.9813091213196 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToStream<Location>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 560.3547097686743 < 597.1404899595108. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 82.98553933166448 (T) = (0 -552.6932792910807) / Math.Sqrt((29.49655858087713 / (299)) + (12.290545365710399 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13301381077718524 = (637.4879855774022 - 552.6932792910807) / 637.4879855774022 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromStream(Mode%3a%20Reflection).html>) 87.34 μs 78.32 μs 0.90 0.13 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 84.66 μs 79.61 μs 0.94 0.14 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20SourceGen).html>) 85.22 μs 76.92 μs 0.90 0.12 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 78.31648869346733 < 83.8398615798264. IsChangePoint: Marked as a change because one of 3/25/2023 3:46:49 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.934646933563513 (T) = (0 -79693.9092463198) / Math.Sqrt((11891115.649452716 / (299)) + (7389297.7531531565 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1275102782708896 = (91340.80008229951 - 79693.9092463198) / 91340.80008229951 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 79.61218602825747 < 81.56221848549616. IsChangePoint: Marked as a change because one of 2/28/2023 3:00:37 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.50357755073757 (T) = (0 -77184.56201524305) / Math.Sqrt((8969502.176121917 / (299)) + (4985634.164838831 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13356789966743032 = (89083.220699714 - 77184.56201524305) / 89083.220699714 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 76.91543898809523 < 81.03574713372979. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.954782817116993 (T) = (0 -77755.99695337577) / Math.Sqrt((4784197.350087582 / (299)) + (4779593.9625241095 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1311856015132165 = (89496.6716583008 - 77755.99695337577) / 89496.6716583008 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_FromString(IndexViewModel).SystemTextJsonReflection.html>) 34.78 μs 28.19 μs 0.81 0.13 False
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_FromString(IndexViewModel).SystemTextJsonSourceGen.html>) 32.78 μs 26.32 μs 0.80 0.02 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonFromString(IndexViewModel).JsonNet.html>) 80.90 μs 61.90 μs 0.77 0.01 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 28.193584544328036 < 32.82575553179667. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 30.986157412935952 (T) = (0 -27218.702055032812) / Math.Sqrt((1068168.773569692 / (299)) + (453571.4585540004 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1825152625399451 = (33295.67000798325 - 27218.702055032812) / 33295.67000798325 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 26.322803967602784 < 31.30044458654925. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.705572573897957 (T) = (0 -26850.411605866797) / Math.Sqrt((366467.3017766039 / (299)) + (428733.1622434331 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17924387579341963 = (32714.238510035993 - 26850.411605866797) / 32714.238510035993 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.JsonNet_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 61.90361284410137 < 76.74366596826391. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 66.03713740249468 (T) = (0 -62056.10433842149) / Math.Sqrt((1189046.1036982439 / (299)) + (1090905.3238286634 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2398245215420624 = (81633.92018946217 - 62056.10433842149) / 81633.92018946217 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Decode_NoDecodingRequired - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired.html>) 91.22 ns 77.74 ns 0.85 0.01 False
[Decode_DecodingRequired - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Net.Tests.Perf_WebUtility.Decode_DecodingRequired.html>) 131.49 ns 118.60 ns 0.90 0.02 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Tests.Perf_WebUtility*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 77.73524786643438 < 86.62232590996726. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.84588943698462 (T) = (0 -75.72469865244042) / Math.Sqrt((25.171751729351435 / (299)) + (1.506903202048684 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09050688578556437 = (83.26033201235039 - 75.72469865244042) / 83.26033201235039 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Net.Tests.Perf_WebUtility.Decode_DecodingRequired ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 118.60398492405724 < 124.71436175405775. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.591552097539541 (T) = (0 -111.24105907310962) / Math.Sqrt((23.216290593251486 / (299)) + (12.013536742875635 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1089637781539233 = (124.84459817204392 - 111.24105907310962) / 124.84459817204392 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Version.Parse3.html>) 45.07 ns 38.68 ns 0.86 0.03 False
[Parse4 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Version.Parse4.html>) 57.79 ns 48.28 ns 0.84 0.01 False
[Parse2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Version.Parse2.html>) 31.65 ns 26.90 ns 0.85 0.04 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Version.Parse3 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 38.68274173269247 < 42.964976122286636. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.258172056078717 (T) = (0 -38.85567957977936) / Math.Sqrt((6.308619461810621 / (299)) + (0.49280072638997047 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06026588405007847 = (41.34752470969138 - 38.85567957977936) / 41.34752470969138 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Version.Parse4 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 48.278348474606126 < 54.79410680597995. IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 61.39219491000591 (T) = (0 -48.35928421669626) / Math.Sqrt((5.9275630692128445 / (299)) + (0.3523629125657369 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2156953912251513 = (61.65880408663877 - 48.35928421669626) / 61.65880408663877 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Version.Parse2 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 26.898471204896882 < 30.398947764859326. IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 61.91395545878987 (T) = (0 -27.095748160447727) / Math.Sqrt((2.462412973166196 / (299)) + (0.11488711248342741 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2299203966378606 = (35.1856457983677 - 27.095748160447727) / 35.1856457983677 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 225.61 ns 183.02 ns 0.81 0.01 False
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(%2c%20None%2c%20False)).html>) 7.16 ns 5.68 ns 0.79 0.36 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 226.34 ns 183.91 ns 0.81 0.01 False
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 227.49 ns 182.68 ns 0.80 0.01 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 54.88 ns 47.12 ns 0.86 0.04 False
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20IgnoreNonSpace%2c%20False)).html>) 7.79 ns 5.68 ns 0.73 0.44 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 226.68 ns 184.30 ns 0.81 0.01 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 183.01760923543006 < 215.7290307037969. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.85078847777229 (T) = (0 -182.8274967923397) / Math.Sqrt((313.2429998103119 / (299)) + (0.0831284617188394 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2419895650081844 = (241.1939049286488 - 182.8274967923397) / 241.1939049286488 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.678164881850679 < 6.792432430275862. IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 3/3/2023 4:19:26 AM, 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.513875594399721 (T) = (0 -6.117477657483776) / Math.Sqrt((0.4653439565688804 / (299)) + (0.18792443866367606 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16443331152989057 = (7.321351774667613 - 6.117477657483776) / 7.321351774667613 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 183.91087977075964 < 215.5393808770308. IsChangePoint: Marked as a change because one of 2/16/2023 5:39:51 AM, 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.52409179071286 (T) = (0 -183.8932117412202) / Math.Sqrt((304.69594882313163 / (299)) + (0.25469334089423873 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23530738528973913 = (240.4799107559011 - 183.8932117412202) / 240.4799107559011 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 182.68014658121507 < 215.65838713682362. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.31319419179072 (T) = (0 -182.93862508032865) / Math.Sqrt((321.1283465186992 / (299)) + (0.14433630252361968 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24280156007837372 = (241.5993158930223 - 182.93862508032865) / 241.5993158930223 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, Ordinal, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 47.1228044949407 < 52.29504474174676. IsChangePoint: Marked as a change because one of 2/16/2023 5:39:51 AM, 3/25/2023 12:18:49 AM, 4/9/2023 11:04:55 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.79738394868738 (T) = (0 -47.17976142063995) / Math.Sqrt((7.255711919186959 / (299)) + (0.4877733310385586 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16971642799768566 = (56.823672070086964 - 47.17976142063995) / 56.823672070086964 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.675580702502313 < 7.024973648618502. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.51930153800556 (T) = (0 -5.997536209350783) / Math.Sqrt((0.4462891094789563 / (299)) + (0.03060752705192338 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1679576090960721 = (7.208209912039555 - 5.997536209350783) / 7.208209912039555 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 184.2967118019224 < 215.55111904965855. IsChangePoint: Marked as a change because one of 2/16/2023 5:39:51 AM, 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.03164072587212 (T) = (0 -184.12818704328976) / Math.Sqrt((297.7121813556958 / (299)) + (0.11791678787539299 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23373508809384733 = (240.2931208023794 - 184.12818704328976) / 240.2931208023794 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryGetValueMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.TryGetValueMiss.html>) 65.46 ns 58.13 ns 0.89 0.11 False
[GetMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.GetMiss.html>) 66.81 ns 60.46 ns 0.90 0.10 False
[SetOverride - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.SetOverride.html>) 172.26 ns 137.15 ns 0.80 0.14 False
[AddThenRemove_SlidingExpiration - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_SlidingExpiration.html>) 32.56 μs 29.22 μs 0.90 0.23 False
[AddThenRemove_ExpirationTokens - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_ExpirationTokens.html>) 43.62 μs 39.63 μs 0.91 0.18 False
[TryGetValueHit - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.TryGetValueHit.html>) 66.70 ns 58.55 ns 0.88 0.11 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.TryGetValueMiss ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 58.13496039142592 < 62.20112145049266. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.235068087379833 (T) = (0 -60.342188068588) / Math.Sqrt((2.514936304320598 / (299)) + (1.8851824207503416 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08458705057771981 = (65.91799701617738 - 60.342188068588) / 65.91799701617738 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.GetMiss ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 60.45978676166873 < 61.46303199744226. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.347654704857916 (T) = (0 -60.94388831806334) / Math.Sqrt((2.7495302445202197 / (299)) + (0.3723397214316162 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06935191972321131 = (65.48542849831884 - 60.94388831806334) / 65.48542849831884 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.SetOverride ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 137.15326789540748 < 158.62824772401135. IsChangePoint: Marked as a change because one of 2/7/2023 10:47:28 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.63026642958669 (T) = (0 -138.6105547475887) / Math.Sqrt((31.624203213769217 / (299)) + (2.4550803943147783 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.16272325994681494 = (165.5492719632746 - 138.6105547475887) / 165.5492719632746 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_SlidingExpiration ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 29.223519818976282 < 30.919206845788235. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.49586066461076 (T) = (0 -29063.094987794837) / Math.Sqrt((1574260.7140540495 / (299)) + (55035.87693736485 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12215491348878162 = (33107.316352705275 - 29063.094987794837) / 33107.316352705275 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_ExpirationTokens ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 39.62900312726614 < 41.51205362178848. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.227132325432557 (T) = (0 -39813.812500513326) / Math.Sqrt((2276297.1766863605 / (299)) + (190866.06339507934 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09878376262564453 = (44177.86858402452 - 39813.812500513326) / 44177.86858402452 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.TryGetValueHit ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 58.54900880241415 < 62.90014280881324. IsChangePoint: Marked as a change because one of 2/2/2023 8:18:56 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.559496962608065 (T) = (0 -60.285246819195656) / Math.Sqrt((3.1356888636254476 / (299)) + (1.5408934476953924 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08980393993633141 = (66.23325398154182 - 60.285246819195656) / 66.23325398154182 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 16.82 μs 15.38 μs 0.91 0.09 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 14.90 μs 13.72 μs 0.92 0.00 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToString(Mode%3a%20Reflection).html>) 19.02 μs 16.38 μs 0.86 0.06 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToString(Mode%3a%20SourceGen).html>) 14.45 μs 13.53 μs 0.94 0.00 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeObjectProperty(Mode%3a%20Reflection).html>) 19.87 μs 17.96 μs 0.90 0.08 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToWriter(Mode%3a%20Reflection).html>) 15.59 μs 13.83 μs 0.89 0.08 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToStream(Mode%3a%20Reflection).html>) 17.58 μs 15.27 μs 0.87 0.12 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 12.08 μs 11.28 μs 0.93 0.00 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToWriter(Mode%3a%20SourceGen).html>) 11.14 μs 9.99 μs 0.90 0.00 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToStream(Mode%3a%20SourceGen).html>) 16.46 μs 13.80 μs 0.84 0.09 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.383204555860805 < 16.06350215830314. IsChangePoint: Marked as a change because one of 3/13/2023 3:41:35 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 27.982536520572147 (T) = (0 -15118.280837043778) / Math.Sqrt((194059.37572547706 / (299)) + (84489.86072562246 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.135314046763574 = (17484.129099654838 - 15118.280837043778) / 17484.129099654838 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.717441622883829 < 14.145436727306027. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 97.76175887745612 (T) = (0 -13772.390253421308) / Math.Sqrt((26629.797056125844 / (299)) + (693.2731583237322 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07808915953680007 = (14938.961176008715 - 13772.390253421308) / 14938.961176008715 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 16.378243364151505 < 18.312473196977873. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.55984247764249 (T) = (0 -16928.576062509288) / Math.Sqrt((235204.3938171879 / (299)) + (275070.3788481281 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1397232579368765 = (19678.058506976515 - 16928.576062509288) / 19678.058506976515 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.529454279556651 < 13.731841798434433. IsChangePoint: Marked as a change because one of 3/17/2023 1:20:12 AM, 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 50.26144585074837 (T) = (0 -13540.165680704933) / Math.Sqrt((27267.535588201914 / (299)) + (4620.765292540639 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07274274484263593 = (14602.383109320663 - 13540.165680704933) / 14602.383109320663 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.957066628484156 < 18.51997575626835. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.81077272398388 (T) = (0 -17468.514779283407) / Math.Sqrt((181374.44458701406 / (299)) + (244404.89119440713 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1304448042047771 = (20089.0235188672 - 17468.514779283407) / 20089.0235188672 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.830934481813822 < 15.459871247356238. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 26.490093536934552 (T) = (0 -13588.792956555757) / Math.Sqrt((166530.27605223437 / (299)) + (145663.86816108227 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17452071513734177 = (16461.700742516678 - 13588.792956555757) / 16461.700742516678 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.26729105809802 < 16.269394257734646. IsChangePoint: Marked as a change because one of 3/13/2023 10:19:02 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.832547762912558 (T) = (0 -14668.426370437064) / Math.Sqrt((184440.75108488864 / (299)) + (178876.2394682962 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15143718356985336 = (17286.199779700768 - 14668.426370437064) / 17286.199779700768 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.281015392317965 < 11.623399313430733. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 84.62263808882675 (T) = (0 -11306.111572983616) / Math.Sqrt((27115.890971999805 / (299)) + (816.4887364434384 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08486098730726745 = (12354.529111064967 - 11306.111572983616) / 12354.529111064967 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.992985400682013 < 10.59288723395243. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 120.58922722813736 (T) = (0 -9989.063931075101) / Math.Sqrt((24832.428834646595 / (299)) + (247.10290971066038 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10870107916137223 = (11207.310698498703 - 9989.063931075101) / 11207.310698498703 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.800801282051282 < 15.932725899860836. IsChangePoint: Marked as a change because one of 2/24/2023 4:59:28 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 16.236235841555164 (T) = (0 -14527.39919576242) / Math.Sqrt((166647.25772682214 / (299)) + (296607.8200326602 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14593192438049166 = (17009.65017949512 - 14527.39919576242) / 17009.65017949512 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.ToStringWithFormat(value%3a%203.4028235E%2b38%2c%20format%3a%20%22E%22).html>) 142.09 ns 131.31 ns 0.92 0.01 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.TryParse(value%3a%20%2212345%22).html>) 63.52 ns 57.07 ns 0.90 0.02 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22F50%22).html>) 314.73 ns 264.48 ns 0.84 0.09 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.Parse(value%3a%20%22-3.4028235E%2b38%22).html>) 103.80 ns 96.88 ns 0.93 0.01 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22E%22).html>) 147.61 ns 136.21 ns 0.92 0.00 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.Parse(value%3a%20%2212345%22).html>) 67.84 ns 59.92 ns 0.88 0.08 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.ToStringWithFormat(value%3a%20-3.4028235E%2b38%2c%20format%3a%20%22E%22).html>) 143.95 ns 134.09 ns 0.93 0.01 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22G17%22).html>) 178.36 ns 150.07 ns 0.84 0.01 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.ToStringWithFormat(value%3a%20-3.4028235E%2b38%2c%20format%3a%20%22F50%22).html>) 1.34 μs 1.19 μs 0.89 0.01 True
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Single.ToStringWithFormat(value%3a%203.4028235E%2b38%2c%20format%3a%20%22F50%22).html>) 1.33 μs 1.19 μs 0.89 0.00 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 131.3131852865112 < 134.90396662197227. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/25/2023 6:46:56 PM, 5/10/2023 6:05:58 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 73.17452690515756 (T) = (0 -131.38390316508247) / Math.Sqrt((15.319460507398816 / (299)) + (0.37903745704030317 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13637924480019278 = (152.13147944167403 - 131.38390316508247) / 152.13147944167403 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Single.TryParse(value: "12345") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 57.067970829973945 < 60.23141916592729. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.98424602127045 (T) = (0 -57.81430351029492) / Math.Sqrt((1.1911971802418257 / (299)) + (0.3586262121520014 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13084563736031446 = (66.51787760083104 - 57.81430351029492) / 66.51787760083104 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "F50") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 264.4754336091811 < 301.02475851545347. IsChangePoint: Marked as a change because one of 3/1/2023 5:53:50 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 63.96363458391147 (T) = (0 -263.9561245115849) / Math.Sqrt((154.73311477583533 / (299)) + (0.5999979570596896 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15392741684641753 = (311.9781089321394 - 263.9561245115849) / 311.9781089321394 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Single.Parse(value: "-3.4028235E+38") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 96.88447799479903 < 98.61054823043162. IsChangePoint: Marked as a change because one of 2/16/2023 5:39:51 AM, 3/13/2023 2:14:21 PM, 5/3/2023 1:18:01 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 27.938582924052145 (T) = (0 -93.8540744573909) / Math.Sqrt((23.85955451722181 / (299)) + (5.3140209551370825 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17223490359026292 = (113.38249808365184 - 93.8540744573909) / 113.38249808365184 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 136.20777004535893 < 140.038135363482. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 73.30511723309705 (T) = (0 -134.29117002875452) / Math.Sqrt((14.239188478973217 / (299)) + (0.6239283758191735 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14441661900187125 = (156.958600425466 - 134.29117002875452) / 156.958600425466 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Single.Parse(value: "12345") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 59.91535052741592 < 60.99217605127382. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.444729601419105 (T) = (0 -58.84694882724909) / Math.Sqrt((1.8346929306783524 / (299)) + (0.6010353828535243 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1329925837082121 = (67.87363951157297 - 58.84694882724909) / 67.87363951157297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "E") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 134.0872144292235 < 137.0069172913148. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 68.12165160344166 (T) = (0 -133.70671179377564) / Math.Sqrt((16.40779879820418 / (299)) + (0.4424196095369585 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1318807538142064 = (154.01883137741186 - 133.70671179377564) / 154.01883137741186 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G17") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 150.07080288574565 < 169.01127515043072. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 66.96975705736556 (T) = (0 -149.7619105144368) / Math.Sqrt((43.72453331856874 / (299)) + (3.576656995524717 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2249698556739719 = (193.2336588593865 - 149.7619105144368) / 193.2336588593865 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "F50") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1901729910714287 < 1.2704695184860348. IsChangePoint: Marked as a change because one of 4/12/2023 2:59:13 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 79.85513187450802 (T) = (0 -1193.0023716025155) / Math.Sqrt((230.74882814376804 / (299)) + (27.35314996052055 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10194071145260902 = (1328.4227297867988 - 1193.0023716025155) / 1328.4227297867988 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "F50") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1887831104864646 < 1.2644005452680758. IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 56.96213610291233 (T) = (0 -1186.2867291168664) / Math.Sqrt((219.3761923627269 / (299)) + (68.86141792823808 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1054803337413681 = (1326.171770016598 - 1186.2867291168664) / 1326.171770016598 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetCount - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList.GetCount.html>) 11.19 ns 6.16 ns 0.55 0.20 False
[Enumerator - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList.Enumerator.html>) 200.13 ns 133.01 ns 0.66 0.22 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList*'
### Payloads [Baseline]() [Compare]() ### Histogram #### XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList.GetCount ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.156275293228292 < 10.841960522430655. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.456912912326706 (T) = (0 -6.266529412966685) / Math.Sqrt((0.7749705238342482 / (299)) + (2.038192351440061 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.46158222473579363 = (11.63878627501041 - 6.266529412966685) / 11.63878627501041 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList.Enumerator ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 133.0052593845349 < 189.68366257398253. IsChangePoint: Marked as a change because one of 3/1/2023 10:50:04 PM, 3/2/2023 6:22:31 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.87469238592563 (T) = (0 -133.23930765401502) / Math.Sqrt((1789.6491024050108 / (299)) + (57.33103868297172 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3757544276431143 = (213.44053294757074 - 133.23930765401502) / 213.44053294757074 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForEachNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ArrayList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEachNonGeneric(Int32).ArrayList(Size%3a%20512).html>) 4.14 μs 2.34 μs 0.56 0.12 False
[Hashtable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEachNonGeneric(Int32).Hashtable(Size%3a%20512).html>) 13.20 μs 11.34 μs 0.86 0.24 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEachNonGeneric(Int32).Queue(Size%3a%20512).html>) 5.19 μs 2.64 μs 0.51 0.11 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForEachNonGeneric(Int32).Stack(Size%3a%20512).html>) 4.18 μs 2.35 μs 0.56 0.17 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEachNonGeneric<Int32>.ArrayList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.3390986618326397 < 3.9332554779882116. IsChangePoint: Marked as a change because one of 2/20/2023 2:00:07 PM, 2/21/2023 5:46:41 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.806256166578237 (T) = (0 -2328.908218104971) / Math.Sqrt((1324106.2560650515 / (299)) + (168.48497090542546 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.47648527519324274 = (4448.601171561375 - 2328.908218104971) / 4448.601171561375 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEachNonGeneric<Int32>.Hashtable(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.342778799336994 < 12.552151343941972. IsChangePoint: Marked as a change because one of 2/20/2023 2:00:07 PM, 2/23/2023 8:54:44 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 7.1135599673618675 (T) = (0 -11840.055641655099) / Math.Sqrt((3007987.041288542 / (299)) + (962577.2041390846 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14838392934846953 = (13903.043929874224 - 11840.055641655099) / 13903.043929874224 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEachNonGeneric<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.644239495205866 < 4.92817665258558. IsChangePoint: Marked as a change because one of 1/22/2023 4:26:49 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 60.57520268786411 (T) = (0 -2658.0754651661246) / Math.Sqrt((516239.28416634875 / (299)) + (2864.0920590624905 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5013791148383033 = (5330.854651834617 - 2658.0754651661246) / 5330.854651834617 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.IterateForEachNonGeneric<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.3524135611546972 < 3.9656849579168205. IsChangePoint: Marked as a change because one of 2/18/2023 8:58:19 AM, 2/19/2023 10:06:54 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 27.080327935028755 (T) = (0 -2351.269942312152) / Math.Sqrt((2120790.4751038444 / (299)) + (2043.708075862982 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4951215609137552 = (4657.101116394676 - 2351.269942312152) / 4657.101116394676 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_FromString(CollectionsOfPrimitives).SystemTextJsonReflection.html>) 340.07 μs 291.27 μs 0.86 0.07 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonFromString(CollectionsOfPrimitives).JsonNet.html>) 700.11 μs 535.98 μs 0.77 0.02 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 291.27303807303804 < 322.09569462243877. IsChangePoint: Marked as a change because one of 2/8/2023 4:45:44 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 6.046872774633721 (T) = (0 -302218.5760309415) / Math.Sqrt((52831633.03165637 / (299)) + (633513771.581337 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1227509486994287 = (344507.15630058007 - 302218.5760309415) / 344507.15630058007 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromString<CollectionsOfPrimitives>.JsonNet_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 535.9821658986176 < 662.4286702889855. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 72.74022501537961 (T) = (0 -543696.1774260699) / Math.Sqrt((97236262.86979952 / (299)) + (66814151.6443029 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2382430058413098 = (713739.6592289199 - 543696.1774260699) / 713739.6592289199 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonFromString(Location).JsonNet.html>) 3.16 μs 2.23 μs 0.70 0.02 True
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_FromString(Location).SystemTextJsonSourceGen.html>) 1.47 μs 1.12 μs 0.76 0.01 True
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_FromString(Location).SystemTextJsonReflection.html>) 1.50 μs 1.14 μs 0.76 0.01 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromString<Location>.JsonNet_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.2256319993375864 < 3.0926472643003913. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.67557118136044 (T) = (0 -2240.1228231565615) / Math.Sqrt((3004.567843227374 / (299)) + (4516.876901156825 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.31969458580572435 = (3292.819337340812 - 2240.1228231565615) / 3292.819337340812 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1213074498913926 < 1.400422631511678. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 76.2240918531332 (T) = (0 -1146.2839741283688) / Math.Sqrt((553.6864286659122 / (299)) + (201.6974472499842 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21698601270196868 = (1463.9380556711171 - 1146.2839741283688) / 1463.9380556711171 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1411245163174417 < 1.4444257241304197. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 78.75452980844717 (T) = (0 -1152.6243026939992) / Math.Sqrt((652.6701358863403 / (299)) + (217.46088272787878 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22906479319615353 = (1495.0988001606056 - 1152.6243026939992) / 1495.0988001606056 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Numerics.Tests.Perf_VectorOf(Int32).GetHashCodeBenchmark.html>) 20.96 ns 18.15 ns 0.87 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int32>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.15185650823174 < 20.144307697011822. IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 2/1/2023 6:25:50 PM, 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 46.468148687775354 (T) = (0 -18.23979511858279) / Math.Sqrt((0.3418338902211805 / (299)) + (0.00983837721394237 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09995049401305313 = (20.265324293003186 - 18.23979511858279) / 20.265324293003186 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromReader(Mode%3a%20SourceGen).html>) 60.98 μs 45.58 μs 0.75 0.17 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromStream(Mode%3a%20SourceGen).html>) 49.45 μs 41.33 μs 0.84 0.26 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromReader(Mode%3a%20Reflection).html>) 54.77 μs 47.33 μs 0.86 0.21 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 48.23 μs 45.01 μs 0.93 0.23 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 51.18 μs 46.26 μs 0.90 0.22 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20Reflection).html>) 49.22 μs 41.05 μs 0.83 0.24 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 45.584749999999985 < 52.581141082875455. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.27228188616367 (T) = (0 -47349.78452281622) / Math.Sqrt((2193204.8155470095 / (299)) + (2571871.2150689983 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14180066683769946 = (55173.410993389276 - 47349.78452281622) / 55173.410993389276 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 41.33031249999999 < 47.48882551107128. IsChangePoint: Marked as a change because one of 2/2/2023 2:17:49 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.75134072796723 (T) = (0 -43356.484508879585) / Math.Sqrt((2947975.6544178496 / (299)) + (5874707.642423875 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14420675138302266 = (50662.335299964965 - 43356.484508879585) / 50662.335299964965 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 47.33029994979918 < 51.977311119218975. IsChangePoint: Marked as a change because one of 2/8/2023 4:45:44 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 27.545073217238684 (T) = (0 -46813.645580958706) / Math.Sqrt((3732659.472553123 / (299)) + (1148486.5884996834 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1574246705551732 = (55560.19022275935 - 46813.645580958706) / 55560.19022275935 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 45.01144345238095 < 45.84250344127323. IsChangePoint: Marked as a change because one of 2/8/2023 4:01:29 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 21.412917596707985 (T) = (0 -41988.62450846282) / Math.Sqrt((3587641.3932201555 / (299)) + (1265993.3662986867 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14432107680514436 = (49070.537289488835 - 41988.62450846282) / 49070.537289488835 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 46.26220588235294 < 47.29968516573295. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.238528250344224 (T) = (0 -42098.16193984995) / Math.Sqrt((2862609.015204567 / (299)) + (3995240.5514878365 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14063820579363823 = (48987.70485686819 - 42098.16193984995) / 48987.70485686819 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 41.04737939927184 < 46.59517783469126. IsChangePoint: Marked as a change because one of 2/9/2023 1:20:08 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.11825847986378 (T) = (0 -42378.53287783009) / Math.Sqrt((2784104.8080055374 / (299)) + (4602320.223778966 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14702316225368667 = (49683.09923842742 - 42378.53287783009) / 49683.09923842742 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 30.50 μs 18.58 μs 0.61 0.16 True
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).ImmutableDictionary(Size%3a%20512).html>) 21.49 μs 13.01 μs 0.61 0.05 True
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).SortedList(Size%3a%20512).html>) 28.35 μs 22.36 μs 0.79 0.02 True
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).SortedDictionary(Size%3a%20512).html>) 39.93 μs 28.56 μs 0.72 0.12 False
[IDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsKeyTrue(Int32%2c%20Int32).IDictionary(Size%3a%20512).html>) 6.47 μs 5.65 μs 0.87 0.03 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.580644757903165 < 29.1897616237426. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 73.97248218756087 (T) = (0 -18985.59241183348) / Math.Sqrt((5082476.011900468 / (299)) + (132915.77274712562 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3913020824360173 = (31190.500023088756 - 18985.59241183348) / 31190.500023088756 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.006857073238653 < 20.432326257487617. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 106.16385242756158 (T) = (0 -13091.82553781462) / Math.Sqrt((114124.68172727578 / (299)) + (75553.89811938953 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3895677080507289 = (21446.810253122378 - 13091.82553781462) / 21446.810253122378 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsKeyTrue<Int32, Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.362562410841655 < 26.902984159051563. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 33.92074816101778 (T) = (0 -22820.863351597895) / Math.Sqrt((764066.3413892053 / (299)) + (544406.7814626244 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23856982988716943 = (29971.05216912569 - 22820.863351597895) / 29971.05216912569 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsKeyTrue<Int32, Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 28.563737015721504 < 37.91920662136845. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 73.27918216016008 (T) = (0 -28844.30502079754) / Math.Sqrt((8144952.820568766 / (299)) + (32773.052585523576 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.304723808185477 = (41486.11064262108 - 28844.30502079754) / 41486.11064262108 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsKeyTrue<Int32, Int32>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.652057035145324 < 6.13716960736089. IsChangePoint: Marked as a change because one of 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 9.876442236882315 (T) = (0 -5446.205585074295) / Math.Sqrt((43094.6078361509 / (299)) + (292820.7822424988 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2144766068134158 = (6933.218835127252 - 5446.205585074295) / 6933.218835127252 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse_SequentialAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length%3a%20128%2c%20Chunks%3a%2016).html>) 703.46 ns 585.22 ns 0.83 0.19 False
[Parse_SequentialAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length%3a%204096%2c%20Chunks%3a%2016).html>) 701.58 ns 653.49 ns 0.93 0.20 False
[Parse_SequentialAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length%3a%204096%2c%20Chunks%3a%201).html>) 241.08 ns 225.90 ns 0.94 0.18 False
[Parse_SequentialAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length%3a%20128%2c%20Chunks%3a%201).html>) 240.67 ns 221.39 ns 0.92 0.22 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length: 128, Chunks: 16) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 585.2198204627404 < 669.0246628270004. IsChangePoint: Marked as a change because one of 4/8/2023 1:46:22 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.119662121638743 (T) = (0 -586.7504467107094) / Math.Sqrt((1066.3111681676749 / (299)) + (86.61748599176282 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1369844089050337 = (679.8839473644496 - 586.7504467107094) / 679.8839473644496 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length: 4096, Chunks: 16) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 653.4910724179965 < 667.5563359827428. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.222952639814878 (T) = (0 -598.9081835263204) / Math.Sqrt((1084.9496282339774 / (299)) + (600.3122761591935 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11680141201114194 = (678.1127049694467 - 598.9081835263204) / 678.1127049694467 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length: 4096, Chunks: 1) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 225.90404301733523 < 230.4138522583644. IsChangePoint: Marked as a change because one of 1/30/2023 1:22:46 AM, 1/30/2023 9:42:46 PM, 5/7/2023 10:29:39 PM, 5/8/2023 9:53:14 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.498624966217044 (T) = (0 -224.1739618668363) / Math.Sqrt((322.6017893666308 / (299)) + (1.5188939153657015 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08276721754431353 = (244.40247465497302 - 224.1739618668363) / 244.40247465497302 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.AspNetCore.Server.Kestrel.Performance.PipeThroughputBenchmark.Parse_SequentialAsync(Length: 128, Chunks: 1) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 221.39362121730537 < 229.07716631720865. IsChangePoint: Marked as a change because one of 3/23/2023 11:51:53 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 5.353547071598076 (T) = (0 -226.13943394385842) / Math.Sqrt((320.0191985579762 / (299)) + (147.75575496394032 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07705219522574487 = (245.0186595320741 - 226.13943394385842) / 245.0186595320741 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CachedDefaultOptions - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).CachedDefaultOptions.html>) 748.51 ns 558.06 ns 0.75 0.09 True
[NewDefaultOptions - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).NewDefaultOptions.html>) 1.36 μs 1.12 μs 0.83 0.12 False
[CachedJsonSerializerContext - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).CachedJsonSerializerContext.html>) 695.84 ns 512.20 ns 0.74 0.08 False
[NewCustomConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).NewCustomConverter.html>) 34.62 μs 31.10 μs 0.90 0.39 False
[NewCachedCustomConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).NewCachedCustomConverter.html>) 1.61 μs 1.38 μs 0.85 0.16 False
[NewCustomizedOptions - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).NewCustomizedOptions.html>) 1.54 μs 1.22 μs 0.79 0.16 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedDefaultOptions ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 558.063122646456 < 696.0791539563509. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 45.07599332157309 (T) = (0 -564.8327775216954) / Math.Sqrt((388.06959339102707 / (299)) + (189.73378918133554 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24135833184514321 = (744.5317087518579 - 564.8327775216954) / 744.5317087518579 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewDefaultOptions ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1217231506734944 < 1.2971206417670325. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/12/2023 6:21:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.3511638011492 (T) = (0 -1123.3205098254361) / Math.Sqrt((1968.7735197538839 / (299)) + (409.3150042337686 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17023494658824898 = (1353.7814170489237 - 1123.3205098254361) / 1353.7814170489237 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedJsonSerializerContext ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 512.2013058000153 < 655.1577034569345. IsChangePoint: Marked as a change because one of 2/21/2023 12:38:54 AM, 4/12/2023 2:59:13 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 59.761405262730136 (T) = (0 -518.950061865187) / Math.Sqrt((434.43711042811265 / (299)) + (78.754087489822 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23989321367724709 = (682.733099089623 - 518.950061865187) / 682.733099089623 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomConverter ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 31.095698635057474 < 31.98672811210926. IsChangePoint: Marked as a change because one of 2/2/2023 4:02:34 PM, 4/18/2023 7:36:21 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 8.544521101899186 (T) = (0 -29015.955870044396) / Math.Sqrt((2792296.166463561 / (299)) + (1292524.3470962464 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08851974648767338 = (31833.883134860465 - 29015.955870044396) / 31833.883134860465 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCachedCustomConverter ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.3779847436579744 < 1.5237208408393355. IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 4/19/2023 4:06:12 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.215519699600826 (T) = (0 -1350.5379074066261) / Math.Sqrt((3983.329356116281 / (299)) + (2123.6139159505415 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1520229046710904 = (1592.6584749117374 - 1350.5379074066261) / 1592.6584749117374 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomizedOptions ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2170534066441094 < 1.4285437567244654. IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 4/11/2023 8:14:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 35.216693492954505 (T) = (0 -1205.3074987871917) / Math.Sqrt((3173.482401690827 / (299)) + (542.4325102078291 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17449536801785293 = (1460.0856883056938 - 1205.3074987871917) / 1460.0856883056938 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LinqQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(IntStruct).LinqQuery(Size%3a%20512).html>) 35.09 μs 22.82 μs 0.65 0.32 False
[Array_Comparison - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(IntStruct).Array_Comparison(Size%3a%20512).html>) 21.26 μs 15.40 μs 0.72 0.29 False
[LinqOrderByExtension - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(IntStruct).LinqOrderByExtension(Size%3a%20512).html>) 34.54 μs 22.51 μs 0.65 0.33 False
[Array_ComparerClass - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(IntStruct).Array_ComparerClass(Size%3a%20512).html>) 22.78 μs 15.02 μs 0.66 0.40 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<IntStruct>.LinqQuery(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.818545333333336 < 32.95390911538461. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 87.89816799581445 (T) = (0 -22777.178303888417) / Math.Sqrt((4153381.8344906135 / (299)) + (143194.13787934565 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.378503126761227 = (36648.90248794164 - 22777.178303888417) / 36648.90248794164 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Sort<IntStruct>.Array_Comparison(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.402782857142855 < 20.30513786666667. IsChangePoint: Marked as a change because one of 2/20/2023 3:29:46 AM, 3/26/2023 5:20:11 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.09471945524303 (T) = (0 -14905.861326852635) / Math.Sqrt((3576504.7916533225 / (299)) + (621546.0767056798 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3448701797847646 = (22752.530669349602 - 14905.861326852635) / 22752.530669349602 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Sort<IntStruct>.LinqOrderByExtension(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.514176 < 33.443705542857145. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 99.14146611655731 (T) = (0 -22610.014202592276) / Math.Sqrt((4226976.337517474 / (299)) + (72851.68120188438 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.38121993925332903 = (36539.661887794464 - 22610.014202592276) / 36539.661887794464 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Sort<IntStruct>.Array_ComparerClass(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.023082352941177 < 21.52265515934066. IsChangePoint: Marked as a change because one of 3/26/2023 5:20:11 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 58.07453024076788 (T) = (0 -14993.784227205677) / Math.Sqrt((3665335.94432784 / (299)) + (72816.15381929655 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.341074974774201 = (22754.916952907715 - 14993.784227205677) / 22754.916952907715 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20BasicJson).html>) 1.01 μs 826.57 ns 0.82 0.00 True
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20BasicJson).html>) 1.90 μs 1.70 μs 0.89 0.00 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20Json400B).html>) 2.94 μs 2.74 μs 0.93 0.00 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20BasicJson).html>) 1.63 μs 1.47 μs 0.90 0.00 True
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 263.71 ns 208.21 ns 0.79 0.12 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20Json400B).html>) 1.58 μs 1.35 μs 0.85 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20Json400KB).html>) 1.22 ms 997.07 μs 0.82 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20Json400B).html>) 2.66 μs 2.39 μs 0.90 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20Json400B).html>) 1.34 μs 1.03 μs 0.77 0.00 True
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 273.13 ns 220.74 ns 0.81 0.07 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 331.99 ns 278.25 ns 0.84 0.05 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20Json400KB).html>) 917.22 μs 706.64 μs 0.77 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 349.92 ns 292.02 ns 0.83 0.09 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20BasicJson).html>) 1.26 μs 1.06 μs 0.84 0.00 True

graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: BasicJson) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 826.5745376915336 < 0.9524980197791594. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 95.75140626061874 (T) = (0 -833.3224777075379) / Math.Sqrt((230.67880806344616 / (299)) + (22.372150800591914 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1535486638387378 = (984.4895295301146 - 833.3224777075379) / 984.4895295301146 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: BasicJson) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.6975940797153186 < 1.8138139152290522. IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.28923032278663 (T) = (0 -1716.5538177327253) / Math.Sqrt((359.0019707439891 / (299)) + (138.2087157800264 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.09974266807103362 = (1906.7368371826465 - 1716.5538177327253) / 1906.7368371826465 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.7393553068861896 < 2.7974083293474714. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 3/8/2023 10:34:21 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.631344922803024 (T) = (0 -2705.026896155455) / Math.Sqrt((1270.801314415103 / (299)) + (230.98374738751485 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07779680306502425 = (2933.2222065005326 - 2705.026896155455) / 2933.2222065005326 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: BasicJson) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.4675262453013465 < 1.561436974176771. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 44.644479313239344 (T) = (0 -1481.3550969036417) / Math.Sqrt((339.14344898499104 / (299)) + (155.7831495726621 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0984110770033088 = (1643.0493533349213 - 1481.3550969036417) / 1643.0493533349213 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 208.20977631047026 < 244.63562109337693. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/11/2023 12:48:48 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 31.952358045262436 (T) = (0 -210.75342992765647) / Math.Sqrt((62.11278630184717 / (299)) + (14.185969985451925 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14733478336067532 = (247.1701974173583 - 210.75342992765647) / 247.1701974173583 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.3477827945402299 < 1.5167076023637527. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 102.16431119576632 (T) = (0 -1337.9203419729647) / Math.Sqrt((348.778214467487 / (299)) + (63.66847550823467 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15827761020591602 = (1589.5030929381228 - 1337.9203419729647) / 1589.5030929381228 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 997.0699049954031 < 1.162096490268521. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 206.74020937427088 (T) = (0 -999223.282023135) / Math.Sqrt((111212560.7517796 / (299)) + (8814273.267535256 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17492243134056845 = (1211065.8682027336 - 999223.282023135) / 1211065.8682027336 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.3890570435937715 < 2.5209429219898607. IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 4/22/2023 11:48:08 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 37.5581542640982 (T) = (0 -2415.3709095607287) / Math.Sqrt((992.5049913454043 / (299)) + (444.6855841248393 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.08697038644906376 = (2645.4464058037693 - 2415.3709095607287) / 2645.4464058037693 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400B) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0317157501484464 < 1.2504697800794378. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 122.7695623666276 (T) = (0 -1032.1636754376775) / Math.Sqrt((178.6278237481611 / (299)) + (52.281078933120824 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20358933171990445 = (1296.0193987188884 - 1032.1636754376775) / 1296.0193987188884 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 220.73906429159348 < 258.9989070836433. IsChangePoint: Marked as a change because one of 2/8/2023 4:01:29 AM, 4/7/2023 7:05:26 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 33.72348289697355 (T) = (0 -223.23813416951873) / Math.Sqrt((65.46598715315967 / (299)) + (15.862626803167222 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15342100893598135 = (263.69439417453884 - 223.23813416951873) / 263.69439417453884 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 278.25196988230874 < 317.5305274504649. IsChangePoint: Marked as a change because one of 3/14/2023 1:07:29 AM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 61.80483680669954 (T) = (0 -282.22404863496143) / Math.Sqrt((70.47642879007917 / (299)) + (4.251846037112667 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14110288905730778 = (328.58889037966753 - 282.22404863496143) / 328.58889037966753 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400KB) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 706.6399195171026 < 873.2102484681373. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 363.8997380989251 (T) = (0 -705551.4976803921) / Math.Sqrt((72687033.74835575 / (299)) + (853201.3208338628 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22274457301095646 = (907747.2773829057 - 705551.4976803921) / 907747.2773829057 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 292.0220793004006 < 334.9549909634687. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 2/28/2023 5:58:27 PM, 3/14/2023 8:42:09 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.806448189941285 (T) = (0 -294.602451254872) / Math.Sqrt((139.43588061819844 / (299)) + (9.577210674542195 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15642756952293171 = (349.23195757863317 - 294.602451254872) / 349.23195757863317 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0589471561162511 < 1.2023779014945035. IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 156.620137645333 (T) = (0 -1064.3379856776116) / Math.Sqrt((119.91279056582054 / (299)) + (15.403843972610758 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15634337760104627 = (1261.5772310909458 - 1064.3379856776116) / 1261.5772310909458 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrueComparer(Int32).ImmutableSortedSet(Size%3a%20512).html>) 27.71 μs 19.87 μs 0.72 0.14 False
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrueComparer(Int32).SortedSet(Size%3a%20512).html>) 29.75 μs 19.08 μs 0.64 0.19 False
[ImmutableHashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrueComparer(Int32).ImmutableHashSet(Size%3a%20512).html>) 22.70 μs 11.48 μs 0.51 0.07 True
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.ContainsTrueComparer(Int32).HashSet(Size%3a%20512).html>) 8.00 μs 6.50 μs 0.81 0.01 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrueComparer<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.87332519618334 < 26.490852116578008. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 77.40070243232358 (T) = (0 -19629.93034309655) / Math.Sqrt((2175134.426841178 / (299)) + (47263.58516078261 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2917125817743418 = (27714.639337053017 - 19629.93034309655) / 27714.639337053017 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsTrueComparer<Int32>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.076005881254666 < 28.322167582170884. IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 3/25/2023 7:34:08 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.61863186226772 (T) = (0 -18947.408995654165) / Math.Sqrt((8832925.680643825 / (299)) + (6808.5942860866535 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.37160061195410743 = (30151.8578090506 - 18947.408995654165) / 30151.8578090506 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsTrueComparer<Int32>.ImmutableHashSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.484312375992065 < 21.33872942353474. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 305.66918051023885 (T) = (0 -11409.47875310215) / Math.Sqrt((194336.223317576 / (299)) + (6775.719002484068 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.478308613132824 = (21870.168916564908 - 11409.47875310215) / 21870.168916564908 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.ContainsTrueComparer<Int32>.HashSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.503855385814269 < 7.586117207932517. IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.02893034682886 (T) = (0 -6278.422593378274) / Math.Sqrt((37928.46099912324 / (299)) + (23866.338174765155 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2531227282318553 = (8406.23062275659 - 6278.422593378274) / 8406.23062275659 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array_ComparerClass - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(Int32).Array_ComparerClass(Size%3a%20512).html>) 21.55 μs 13.11 μs 0.61 0.29 False
[LinqQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(Int32).LinqQuery(Size%3a%20512).html>) 34.42 μs 23.12 μs 0.67 0.29 False
[Array_Comparison - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(Int32).Array_Comparison(Size%3a%20512).html>) 25.71 μs 13.15 μs 0.51 0.26 False
[LinqOrderByExtension - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Sort(Int32).LinqOrderByExtension(Size%3a%20512).html>) 34.33 μs 23.10 μs 0.67 0.32 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<Int32>.Array_ComparerClass(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.111657000000001 < 20.42700459047619. IsChangePoint: Marked as a change because one of 4/26/2023 2:24:59 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 82.2020451949993 (T) = (0 -13320.45169482879) / Math.Sqrt((3643422.473847437 / (299)) + (37196.35919292856 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.43084082995860273 = (23403.73729524544 - 13320.45169482879) / 23403.73729524544 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Sort<Int32>.LinqQuery(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.118371999999997 < 32.59425962564102. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 100.2376127086576 (T) = (0 -23248.405785573395) / Math.Sqrt((4328463.233024441 / (299)) + (9103.344525460006 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.34690100382947675 = (35597.062500312386 - 23248.405785573395) / 35597.062500312386 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Sort<Int32>.Array_Comparison(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.145779999999998 < 20.414725123626372. IsChangePoint: Marked as a change because one of 4/17/2023 2:42:45 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 79.42866639953151 (T) = (0 -13295.551002786802) / Math.Sqrt((4576412.61637781 / (299)) + (16562.683093843854 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.43478468496318723 = (23522.984337253594 - 13295.551002786802) / 23522.984337253594 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Sort<Int32>.LinqOrderByExtension(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.097989333333334 < 32.54165235952381. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 72.71749040925391 (T) = (0 -23404.55463623556) / Math.Sqrt((4428059.703397062 / (299)) + (181299.79463212742 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3450638443115422 = (35735.62771417481 - 23404.55463623556) / 35735.62771417481 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Int16.ToString(value%3a%20-32768).html>) 20.54 ns 17.72 ns 0.86 0.06 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Int16.ToString(value%3a%200).html>) 3.52 ns 1.44 ns 0.41 0.55 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Int16.ToString(value%3a%2032767).html>) 13.33 ns 9.89 ns 0.74 0.16 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Int16.TryParse(value%3a%20%220%22).html>) 7.47 ns 5.75 ns 0.77 0.27 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int16.ToString(value: -32768) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.723968915656187 < 19.826851977394167. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 83.06892113969843 (T) = (0 -17.924468040438107) / Math.Sqrt((1.0089438758564067 / (299)) + (0.04562581776973145 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27772621035630696 = (24.816722269931017 - 17.924468040438107) / 24.816722269931017 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Int16.ToString(value: 0) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.4365991144931498 < 3.2112230992009727. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 1:02:54 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.737632418193368 (T) = (0 -1.441261567913527) / Math.Sqrt((0.22260003725878796 / (299)) + (0.028985515154589057 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4396778702361013 = (2.572201759228798 - 1.441261567913527) / 2.572201759228798 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Int16.ToString(value: 32767) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.893171980862773 < 12.716353927128464. IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.892293591519678 (T) = (0 -10.894819026896688) / Math.Sqrt((0.7782189449834155 / (299)) + (0.25094940599769894 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1289003369325001 = (12.50697192159569 - 10.894819026896688) / 12.50697192159569 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Int16.TryParse(value: "0") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.748123663766148 < 7.341080131767309. IsChangePoint: Marked as a change because one of 4/25/2023 1:14:10 AM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 60.53561292635478 (T) = (0 -5.67074553633632) / Math.Sqrt((0.6952642431416756 / (299)) + (0.030967326723844706 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4227734235319255 = (9.824124126498809 - 5.67074553633632) / 9.824124126498809 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ImmutableSortedDictionary(String%2c%20String)).SerializeToStream(Mode%3a%20Reflection).html>) 18.54 μs 17.18 μs 0.93 0.22 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ImmutableSortedDictionary(String%2c%20String)).SerializeToWriter(Mode%3a%20Reflection).html>) 18.26 μs 15.21 μs 0.83 0.32 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ImmutableSortedDictionary(String%2c%20String)).SerializeToString(Mode%3a%20Reflection).html>) 21.21 μs 18.62 μs 0.88 0.27 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ImmutableSortedDictionary(String%2c%20String)).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 18.81 μs 15.82 μs 0.84 0.31 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ImmutableSortedDictionary(String%2c%20String)).SerializeObjectProperty(Mode%3a%20Reflection).html>) 21.31 μs 17.08 μs 0.80 0.31 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.17615144418423 < 17.50888673434688. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.080887661399883 (T) = (0 -15746.483207419586) / Math.Sqrt((309638.68403516326 / (299)) + (694627.3077856695 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17266370234528305 = (19032.747930988604 - 15746.483207419586) / 19032.747930988604 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.213335350112938 < 17.38955549362853. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 27.64599368818485 (T) = (0 -15112.342245728178) / Math.Sqrt((318977.97539922904 / (299)) + (220372.45469842237 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19714923200775117 = (18823.35154703879 - 15112.342245728178) / 18823.35154703879 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.61653853794199 < 19.1470803859721. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 15.361817295369875 (T) = (0 -17155.523554103063) / Math.Sqrt((331359.1701257202 / (299)) + (671865.0074517303 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17063260400854396 = (20685.071099997516 - 17155.523554103063) / 20685.071099997516 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.822497502497505 < 18.46928632799999. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.996747198226222 (T) = (0 -16133.894443204314) / Math.Sqrt((320600.29591679445 / (299)) + (649862.0955743288 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17358209658721185 = (19522.68262410281 - 16133.894443204314) / 19522.68262410281 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.082835087132068 < 19.373482002701067. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.264488625469845 (T) = (0 -17206.98195520677) / Math.Sqrt((687244.5997244996 / (299)) + (216410.71904680485 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18968841505992237 = (21235.019065510736 - 17206.98195520677) / 21235.019065510736 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IntegerFormatting - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/PerfLabTests.LowLevelPerf.IntegerFormatting.html>) 1.97 ms 1.46 ms 0.75 0.33 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.IntegerFormatting ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.46454237012987 < 1.9186595250186012. IsChangePoint: Marked as a change because one of 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 39.92547681118295 (T) = (0 -1452866.5957600088) / Math.Sqrt((17240458338.041027 / (299)) + (432995770.28789836 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20767017682599648 = (1833663.902666131 - 1452866.5957600088) / 1833663.902666131 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Ctor

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 19.93 ns 17.14 ns 0.86 0.24 False
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted%3a%20True%2c%20SkipValidation%3a%20True).html>) 19.77 ns 16.95 ns 0.86 0.29 False
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted%3a%20False%2c%20SkipValidation%3a%20True).html>) 19.68 ns 18.27 ns 0.93 0.29 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Ctor*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.135923561937126 < 18.50391563416078. IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.052562121540877 (T) = (0 -17.3086547988895) / Math.Sqrt((0.5513042409021839 / (299)) + (0.31009638523749145 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1499889284159661 = (20.362858058582745 - 17.3086547988895) / 20.362858058582745 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted: True, SkipValidation: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 16.95147486519697 < 18.868928824526037. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.594264979973671 (T) = (0 -17.788961540863024) / Math.Sqrt((0.5383955592637714 / (299)) + (0.6290759047534648 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1274132598756084 = (20.38646786946031 - 17.788961540863024) / 20.38646786946031 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted: False, SkipValidation: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.269526364489014 < 19.134006470605772. IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.916793546994384 (T) = (0 -17.487505590214646) / Math.Sqrt((0.5338142140545097 / (299)) + (0.47254536415809895 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14278965674487207 = (20.400483647698955 - 17.487505590214646) / 20.400483647698955 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromReader(Mode%3a%20SourceGen).html>) 43.81 μs 35.67 μs 0.81 0.06 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 32.27 μs 25.99 μs 0.81 0.10 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromString(Mode%3a%20Reflection).html>) 33.77 μs 27.67 μs 0.82 0.15 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromStream(Mode%3a%20Reflection).html>) 34.16 μs 27.59 μs 0.81 0.12 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromString(Mode%3a%20SourceGen).html>) 32.63 μs 27.74 μs 0.85 0.06 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 32.61 μs 24.61 μs 0.75 0.13 False
[DeserializeFromStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromStream(Mode%3a%20SourceGen).html>) 35.59 μs 26.21 μs 0.74 0.10 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromReader(Mode%3a%20Reflection).html>) 45.17 μs 36.09 μs 0.80 0.11 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 35.67052363430512 < 42.49448797384673. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 48.32695083566987 (T) = (0 -34917.424721400355) / Math.Sqrt((559731.9797358187 / (299)) + (436096.900344553 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20664553993728657 = (44012.388508712975 - 34917.424721400355) / 44012.388508712975 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 25.991329775334762 < 30.79854177063271. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 29.83195726945392 (T) = (0 -25397.892662955626) / Math.Sqrt((483656.3814893234 / (299)) + (564677.1335310211 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19956271866548758 = (31730.02214565958 - 25397.892662955626) / 31730.02214565958 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.666273905223363 < 31.812542012740124. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 30.2518743541411 (T) = (0 -27050.47416774139) / Math.Sqrt((1679091.2924736235 / (299)) + (470384.4761718531 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18609497943168413 = (33235.41873332244 - 27050.47416774139) / 33235.41873332244 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.594015868301582 < 33.165939834444046. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 33.25570867393026 (T) = (0 -27743.813780669923) / Math.Sqrt((1021591.6657627113 / (299)) + (505369.29872655723 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19775678409582856 = (34582.7963772821 - 27743.813780669923) / 34582.7963772821 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.738436351012293 < 31.98831260588054. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.421032187996364 (T) = (0 -27057.435195628063) / Math.Sqrt((477297.7837240338 / (299)) + (833598.395931196 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1879022752578222 = (33317.95468854216 - 27057.435195628063) / 33317.95468854216 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 24.612909883242885 < 31.04567484061904. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.00577488332525 (T) = (0 -25705.97669751612) / Math.Sqrt((1363356.7851083574 / (299)) + (734539.3624412386 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18749715699446612 = (31638.014462111903 - 25705.97669751612) / 31638.014462111903 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 26.21478057986952 < 33.53694788305421. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 35.19229451804565 (T) = (0 -27419.05226239768) / Math.Sqrt((592174.2501310668 / (299)) + (515305.96469017 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2075092431260733 = (34598.576733633294 - 27419.05226239768) / 34598.576733633294 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 36.08708297279108 < 42.93882131941608. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 77.00300310783224 (T) = (0 -35370.16586809229) / Math.Sqrt((1658849.1686529568 / (299)) + (92425.48462634748 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1967438080686972 = (44033.48050520508 - 35370.16586809229) / 44033.48050520508 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonToStream(CollectionsOfPrimitives).DataContractJsonSerializer.html>) 1.40 ms 991.90 μs 0.71 0.13 True
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_ToStream(CollectionsOfPrimitives).SystemTextJsonSourceGen.html>) 147.76 μs 127.49 μs 0.86 0.00 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonToStream(CollectionsOfPrimitives).JsonNet.html>) 442.92 μs 355.38 μs 0.80 0.15 True
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_ToStream(CollectionsOfPrimitives).SystemTextJsonReflection.html>) 142.85 μs 127.61 μs 0.89 0.00 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 991.8962121212123 < 1.3285205976622398. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 115.76188274411459 (T) = (0 -987802.2564021739) / Math.Sqrt((1458149506.6031747 / (299)) + (102496011.32816775 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29509870250727416 = (1401334.144107413 - 987802.2564021739) / 1401334.144107413 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 127.49357544757032 < 140.39813048095928. IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 106.56571205755486 (T) = (0 -127527.6107588831) / Math.Sqrt((8163116.113570202 / (299)) + (131810.95973688303 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13918572242754532 = (148147.64819946786 - 127527.6107588831) / 148147.64819946786 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.JsonNet_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 355.38070818070815 < 433.2751948083039. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 83.02106412599905 (T) = (0 -357482.5166937477) / Math.Sqrt((127270836.70559411 / (299)) + (13586895.1948866 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21975604241743302 = (458167.6195242027 - 357482.5166937477) / 458167.6195242027 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 127.61310010214504 < 135.76764588435577. IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 80.99276086526035 (T) = (0 -127744.47810421212) / Math.Sqrt((7599103.040984877 / (299)) + (154463.34761956256 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10908755462354487 = (143386.1192165002 - 127744.47810421212) / 143386.1192165002 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableHashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(String).ImmutableHashSet(Size%3a%20512).html>) 159.72 μs 131.20 μs 0.82 0.31 False
[ConcurrentQueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(String).ConcurrentQueue(Size%3a%20512).html>) 8.34 μs 7.91 μs 0.95 0.13 False
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(String).HashSet(Size%3a%20512).html>) 19.50 μs 17.18 μs 0.88 0.01 False
[FrozenDictionaryOptimized - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(String).FrozenDictionaryOptimized(Size%3a%20512).html>) 147.30 μs 108.55 μs 0.74 0.03 False
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.CtorFromCollection(String).ImmutableDictionary(Size%3a%20512).html>) 173.23 μs 152.63 μs 0.88 0.27 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<String>.ImmutableHashSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 131.1955528846154 < 152.7216440313699. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.74180943099735 (T) = (0 -127071.28086857537) / Math.Sqrt((47702576.94509002 / (299)) + (7563732.291017161 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22047930500537685 = (163012.06842167527 - 127071.28086857537) / 163012.06842167527 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<String>.ConcurrentQueue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.910363301442791 < 7.930495569167342. IsChangePoint: Marked as a change because one of 2/28/2023 7:22:28 AM, 3/1/2023 1:48:25 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.660532654060265 (T) = (0 -7866.383609557306) / Math.Sqrt((183406.57948599127 / (299)) + (1300.1693133772944 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06257855611868614 = (8391.51233514268 - 7866.383609557306) / 8391.51233514268 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<String>.HashSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.17916962314727 < 18.114044827345626. IsChangePoint: Marked as a change because one of 5/3/2023 11:00:11 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 54.22571771687424 (T) = (0 -17172.307608641477) / Math.Sqrt((352256.3308098137 / (299)) + (18479.120118002884 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13867433685476316 = (19937.06717844059 - 17172.307608641477) / 19937.06717844059 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<String>.FrozenDictionaryOptimized(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 108.55459280303029 < 140.1179569925088. IsChangePoint: Marked as a change because one of 4/9/2023 11:24:15 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.86444688983913 (T) = (0 -120985.36152593225) / Math.Sqrt((22451364.24757943 / (171)) + (72630507.69249207 / (13))) is greater than 1.9730840773357494 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (171) + (13) - 2, .975) and 0.20272163235522278 = (151747.95458621622 - 120985.36152593225) / 151747.95458621622 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.CtorFromCollection<String>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 152.62580532212885 < 166.4301460764189. IsChangePoint: Marked as a change because one of 1/30/2023 1:22:46 AM, 1/30/2023 9:42:46 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 27.33659693225265 (T) = (0 -142595.27087067123) / Math.Sqrt((66805600.85440607 / (299)) + (19025945.483041555 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19935689097366632 = (178100.91570522863 - 142595.27087067123) / 178100.91570522863 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FrozenDictionaryOptimized - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueFalse(Int32%2c%20Int32).FrozenDictionaryOptimized(Size%3a%20512).html>) 3.10 μs 2.49 μs 0.80 0.01 True
[IDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueFalse(Int32%2c%20Int32).IDictionary(Size%3a%20512).html>) 7.19 μs 3.92 μs 0.55 0.01 True
[ImmutableDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueFalse(Int32%2c%20Int32).ImmutableDictionary(Size%3a%20512).html>) 18.28 μs 14.31 μs 0.78 0.01 True
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueFalse(Int32%2c%20Int32).SortedList(Size%3a%20512).html>) 32.41 μs 27.56 μs 0.85 0.01 True
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueFalse(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 32.64 μs 27.01 μs 0.83 0.23 False
[FrozenDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueFalse(Int32%2c%20Int32).FrozenDictionary(Size%3a%20512).html>) 7.61 μs 4.47 μs 0.59 0.03 True
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueFalse(Int32%2c%20Int32).SortedDictionary(Size%3a%20512).html>) 48.77 μs 30.21 μs 0.62 0.15 False
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueFalse(Int32%2c%20Int32).ConcurrentDictionary(Size%3a%20512).html>) 5.43 μs 3.28 μs 0.61 0.13 False
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.TryGetValueFalse(Int32%2c%20Int32).Dictionary(Size%3a%20512).html>) 6.46 μs 4.02 μs 0.62 0.05 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueFalse<Int32, Int32>.FrozenDictionaryOptimized(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.4857793928457586 < 2.946184527715265. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 189.37301808052723 (T) = (0 -2488.976710133329) / Math.Sqrt((1760.8892772128395 / (174)) + (31.573440788386833 / (13))) is greater than 1.9728699462106976 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (174) + (13) - 2, .975) and 0.21230341309368692 = (3159.8165480300127 - 2488.976710133329) / 3159.8165480300127 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueFalse<Int32, Int32>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.9229005453129058 < 6.835796482521066. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 131.5098535315383 (T) = (0 -3930.583057393529) / Math.Sqrt((241823.4289804915 / (299)) + (3866.046547472759 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5266907644426649 = (8304.47150004406 - 3930.583057393529) / 8304.47150004406 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.313525114155253 < 17.331587650459575. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.83566218798953 (T) = (0 -14033.740276146347) / Math.Sqrt((78778.93583608122 / (299)) + (92522.42633407922 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23376280277221334 = (18315.138350004174 - 14033.740276146347) / 18315.138350004174 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.560264423076923 < 30.79000514158171. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 25.135684489081367 (T) = (0 -24377.567235218146) / Math.Sqrt((351263.13817728416 / (299)) + (998539.1103516441 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22356842022221243 = (31396.92906642841 - 24377.567235218146) / 31396.92906642841 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.011825304187887 < 30.234848651871097. IsChangePoint: Marked as a change because one of 2/20/2023 2:00:07 PM, 2/23/2023 8:54:44 PM, 3/9/2023 3:58:44 PM, 3/10/2023 10:00:23 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.75131650185073 (T) = (0 -24701.457028179204) / Math.Sqrt((20849806.05709762 / (299)) + (1337813.7021713918 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2493814569829207 = (32908.135907344826 - 24701.457028179204) / 32908.135907344826 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueFalse<Int32, Int32>.FrozenDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.470261683357177 < 7.266495147116995. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 146.81427837503406 (T) = (0 -4461.143333543893) / Math.Sqrt((52440.1167911694 / (173)) + (3755.6933356461013 / (13))) is greater than 1.972940542389176 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (173) + (13) - 2, .975) and 0.44467237706176205 = (8033.353914469422 - 4461.143333543893) / 8033.353914469422 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueFalse<Int32, Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 30.213442125160878 < 46.3454854127669. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 75.68157388262144 (T) = (0 -31266.381880452467) / Math.Sqrt((6195063.260853243 / (299)) + (246960.6867313555 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.3254128582606577 = (46348.91468555988 - 31266.381880452467) / 46348.91468555988 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueFalse<Int32, Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.2836051926718346 < 5.160232595640753. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 93.98377518266548 (T) = (0 -3283.4832096204987) / Math.Sqrt((196386.9451261773 / (299)) + (2.6813491149313466 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4231924523817495 = (5692.510826494267 - 3283.4832096204987) / 5692.510826494267 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.01890145656097 < 6.193900770726949. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/5/2023 3:18:41 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.004127082300016 (T) = (0 -4434.487217515773) / Math.Sqrt((233919.95633834557 / (299)) + (70275.59881700222 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2898119778627257 = (6244.103081561996 - 4434.487217515773) / 6244.103081561996 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Numerics.Tests.Perf_VectorOf(Byte).GetHashCodeBenchmark.html>) 75.05 ns 57.08 ns 0.76 0.05 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Byte>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 57.076656418559 < 71.20527243377435. IsChangePoint: Marked as a change because one of 3/9/2023 9:18:02 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 96.3606540766894 (T) = (0 -57.1217005058344) / Math.Sqrt((2.894272136508708 / (299)) + (0.38597852304957586 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25078022323858223 = (76.24158127905943 - 57.1217005058344) / 76.24158127905943 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.DictionarySequentialKeys

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryGetValue_17_Int_32ByteValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteValue.html>) 6.89 ns 3.96 ns 0.57 0.01 True
[TryGetValue_17_Int_32ByteRefsValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteRefsValue.html>) 6.92 ns 3.96 ns 0.57 0.02 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteValue ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.956319815347673 < 6.559541124957474. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 197.21294338925702 (T) = (0 -3.9428880180389103) / Math.Sqrt((0.03495377861682996 / (299)) + (0.002394464294317895 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.46463987563075193 = (7.3649265953162875 - 3.9428880180389103) / 7.3649265953162875 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteRefsValue ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.9578099331954246 < 6.5848247678059755. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 231.93578458857684 (T) = (0 -3.9490441874439997) / Math.Sqrt((0.04154767565310764 / (299)) + (0.0010667700326977027 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4661384686598898 = (7.3971319445531645 - 3.9490441874439997) / 7.3971319445531645 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Append_ValueTypes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Tests.Perf_StringBuilder.Append_ValueTypes.html>) 2.65 μs 2.26 μs 0.85 0.03 False
[ToString_MultipleSegments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length%3a%20100).html>) 37.27 ns 34.72 ns 0.93 0.13 False
[Append_ValueTypes_Interpolated - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Tests.Perf_StringBuilder.Append_ValueTypes_Interpolated.html>) 3.35 μs 2.73 μs 0.82 0.01 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_StringBuilder.Append_ValueTypes ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.2550769246633258 < 2.4996276075044506. IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 52.06526648351441 (T) = (0 -2265.9825329530754) / Math.Sqrt((45253.02914450139 / (299)) + (664.655993480731 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.24639049024821535 = (3006.8390905781152 - 2265.9825329530754) / 3006.8390905781152 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 34.71884014508896 < 36.03746676349468. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 14.355275994058438 (T) = (0 -34.71824440198673) / Math.Sqrt((4.420302000501003 / (299)) + (0.3753219922661249 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.07952109384590106 = (37.717588279176155 - 34.71824440198673) / 37.717588279176155 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Tests.Perf_StringBuilder.Append_ValueTypes_Interpolated ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.7313795138128563 < 3.1458381056206637. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 4/9/2023 1:38:49 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 61.60491439487351 (T) = (0 -2748.4112290380926) / Math.Sqrt((68934.5881907941 / (299)) + (514.7532985042666 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2692258652522108 = (3760.9585484120166 - 2748.4112290380926) / 3760.9585484120166 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%2010).html>) 766.15 ns 682.54 ns 0.89 0.01 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%2010).html>) 729.05 ns 691.70 ns 0.95 0.01 False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%2010).html>) 811.45 ns 709.02 ns 0.87 0.01 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%2010).html>) 678.38 ns 599.54 ns 0.88 0.01 False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%2010).html>) 1.03 μs 886.41 ns 0.86 0.01 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%2010).html>) 884.27 ns 752.43 ns 0.85 0.01 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%2010).html>) 939.53 ns 780.21 ns 0.83 0.01 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 682.5358960838792 < 725.1666372753002. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 42.48250736231765 (T) = (0 -690.9462775817381) / Math.Sqrt((737.167984883986 / (299)) + (26.254191783483154 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1152092992961362 = (780.9149407109278 - 690.9462775817381) / 780.9149407109278 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 691.6998247079056 < 696.1767098842396. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 11.060861425489946 (T) = (0 -682.3528122790891) / Math.Sqrt((737.4417159383751 / (299)) + (221.98446973047115 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.06686659269985264 = (731.2489371196702 - 682.3528122790891) / 731.2489371196702 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 709.0176338335463 < 773.1404623378436. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 38.4249833419322 (T) = (0 -726.084581068203) / Math.Sqrt((756.0151473270275 / (299)) + (66.45695486344552 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1276137982836301 = (832.2971863145854 - 726.084581068203) / 832.2971863145854 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 599.5430273689218 < 644.3713718532368. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 1:14:10 AM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.13033412622032 (T) = (0 -618.7134464813786) / Math.Sqrt((741.2601380607365 / (299)) + (392.7719170215312 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.08855228140379871 = (678.8249439411754 - 618.7134464813786) / 678.8249439411754 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 886.4050630839081 < 0.9726483173051277. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 1:14:10 AM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 7.076025764601346 (T) = (0 -954.6812918545811) / Math.Sqrt((742.0899922519284 / (299)) + (1640.7021590734512 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.08041780569245595 = (1038.1685267117064 - 954.6812918545811) / 1038.1685267117064 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 752.4294350668157 < 840.8166268996981. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.26828608118243 (T) = (0 -758.8884590881955) / Math.Sqrt((1072.7284273389942 / (299)) + (44.61605224131432 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1416499367248768 = (884.1246614377569 - 758.8884590881955) / 884.1246614377569 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 780.2126090374086 < 895.2055142024735. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 1:14:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 60.729981473688404 (T) = (0 -785.8147207253617) / Math.Sqrt((855.4801265352382 / (299)) + (48.07614767953515 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1652267468932672 = (941.3511007939405 - 785.8147207253617) / 941.3511007939405 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).GetHashCodeBenchmark.html>) 12.25 ns 10.58 ns 0.86 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.576172608753575 < 11.679719815486871. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 55.84838063666149 (T) = (0 -10.530883662840749) / Math.Sqrt((0.10778101809541653 / (299)) + (0.009653380876150539 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1497561833905732 = (12.385722138898283 - 10.530883662840749) / 12.385722138898283 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Midpoint

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Benchstone.BenchI.Midpoint.Test.html>) 542.18 ms 419.92 ms 0.77 0.01 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Midpoint*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.Midpoint.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 419.92182 < 515.0744974615384. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.60569831885699 (T) = (0 -424235371.1476307) / Math.Sqrt((70087400246110.31 / (299)) + (211556447329103 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2379630860406909 = (556712363.1103832 - 424235371.1476307) / 556712363.1103832 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[K_Max_Elements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(String%2c%20String).K_Max_Elements(Size%3a%2010).html>) 1.30 μs 1.15 μs 0.89 0.04 False
[HeapSort - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(String%2c%20String).HeapSort(Size%3a%2010).html>) 3.01 μs 2.81 μs 0.93 0.04 False
[Dequeue_And_Enqueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(String%2c%20String).Dequeue_And_Enqueue(Size%3a%2010).html>) 7.31 μs 6.74 μs 0.92 0.05 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_PriorityQueue<String, String>.K_Max_Elements(Size: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1530334325447733 < 1.2116306530171186. IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 1/31/2023 1:54:46 PM, 3/1/2023 10:50:04 PM, 3/3/2023 12:05:09 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 33.645856883229634 (T) = (0 -1162.4894968589774) / Math.Sqrt((1574.5406614373017 / (299)) + (275.1956030698957 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12953388290263804 = (1335.4793185234946 - 1162.4894968589774) / 1335.4793185234946 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Tests.Perf_PriorityQueue<String, String>.HeapSort(Size: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.809223827577418 < 2.857168359241851. IsChangePoint: Marked as a change because one of 3/2/2023 6:22:31 PM, 3/3/2023 12:05:09 PM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.26491813501029 (T) = (0 -2806.2518598579536) / Math.Sqrt((5736.825846696667 / (299)) + (141.25777551661096 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1115391413495288 = (3158.5542936809998 - 2806.2518598579536) / 3158.5542936809998 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Tests.Perf_PriorityQueue<String, String>.Dequeue_And_Enqueue(Size: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.744880179344465 < 6.9421745692239885. IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 3/2/2023 4:26:55 AM, 4/12/2023 8:39:35 PM, 4/17/2023 8:43:19 PM, 5/12/2023 9:12:45 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 70.65522423863109 (T) = (0 -6782.836362255447) / Math.Sqrt((37184.431932335174 / (299)) + (419.3142376111441 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11532817127922433 = (7667.064940977428 - 6782.836362255447) / 7667.064940977428 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.MDBenchI.MDNDhrystone

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Benchstone.MDBenchI.MDNDhrystone.Test.html>) 487.36 ms 439.83 ms 0.90 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDNDhrystone*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.MDBenchI.MDNDhrystone.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 439.8274416666667 < 447.6869641895604. IsChangePoint: Marked as a change because one of 3/26/2023 1:36:36 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 32.29237270667328 (T) = (0 -440942944.2913497) / Math.Sqrt((179176112970545.38 / (299)) + (9989161184718.977 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.0788890468461894 = (478707741.7564042 - 440942944.2913497) / 478707741.7564042 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Runtime.InteropServices.Tests.SafeHandleTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[AddRef_GetHandle_Release - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Runtime.InteropServices.Tests.SafeHandleTests.AddRef_GetHandle_Release.html>) 20.09 ns 15.14 ns 0.75 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.InteropServices.Tests.SafeHandleTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.InteropServices.Tests.SafeHandleTests.AddRef_GetHandle_Release ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.143588464358398 < 19.086664587464238. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 20.67587654622679 (T) = (0 -15.854076066109146) / Math.Sqrt((0.28490152867428015 / (299)) + (0.3291439425857421 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.17449602176469628 = (19.2053296944743 - 15.854076066109146) / 19.2053296944743 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ImmutableDictionary(String%2c%20String)).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 28.33 μs 26.45 μs 0.93 0.53 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 26.445239301801802 < 26.632890005462215. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.10675725114454 (T) = (0 -24136.934838245743) / Math.Sqrt((2123866.014475987 / (299)) + (2006081.6807618157 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14400451361744132 = (28197.502466103593 - 24136.934838245743) / 28197.502466103593 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlToStream(MyEventsListerViewModel).XmlSerializer.html>) 499.77 μs 422.67 μs 0.85 0.07 False
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlToStream(MyEventsListerViewModel).DataContractSerializer.html>) 772.39 μs 628.94 μs 0.81 0.05 False
[DataContractSerializerBinaryXml - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Xml_ToStream(MyEventsListerViewModel).DataContractSerializerBinaryXml.html>) 508.62 μs 355.87 μs 0.70 0.03 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.XmlSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 422.66577683615816 < 486.0327665119559. IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 15.527027979376928 (T) = (0 -436699.44339358906) / Math.Sqrt((118332446.81202401 / (299)) + (203927490.20487309 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1247937316100688 = (498967.45392027526 - 436699.44339358906) / 498967.45392027526 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 628.9355388471175 < 730.7897168054337. IsChangePoint: Marked as a change because one of 4/18/2023 7:36:21 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 63.54910437294361 (T) = (0 -618653.5719594113) / Math.Sqrt((751719790.282913 / (299)) + (70094204.68212138 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.224101409565788 = (797338.1825751191 - 618653.5719594113) / 797338.1825751191 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_BinaryXml_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 355.8651481331169 < 477.59971111751145. IsChangePoint: Marked as a change because one of 4/4/2023 3:37:20 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 43.82491943812457 (T) = (0 -368195.97844881064) / Math.Sqrt((279561456.49194735 / (217)) + (141925554.77789798 / (13))) is greater than 1.9704231946743616 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (217) + (13) - 2, .975) and 0.29370374352398715 = (521305.2951545911 - 368195.97844881064) / 521305.2951545911 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_ToStream(IndexViewModel).SystemTextJsonSourceGen.html>) 16.22 μs 14.84 μs 0.91 0.13 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonToStream(IndexViewModel).JsonNet.html>) 51.59 μs 36.23 μs 0.70 0.02 True
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Json_ToStream(IndexViewModel).SystemTextJsonReflection.html>) 18.01 μs 13.93 μs 0.77 0.17 False
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.JsonToStream(IndexViewModel).DataContractJsonSerializer.html>) 101.29 μs 69.36 μs 0.68 0.03 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.836587992583974 < 15.940885194570402. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 17.66649829575288 (T) = (0 -14197.042256087589) / Math.Sqrt((154013.82952666614 / (299)) + (283523.8539640968 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.15677845142990865 = (16836.66917687586 - 14197.042256087589) / 16836.66917687586 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.JsonNet_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 36.23150214174455 < 48.787831224362606. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 192.31001473601293 (T) = (0 -36561.62181134135) / Math.Sqrt((469768.4767887701 / (299)) + (53105.643138500316 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2834538146346638 = (51024.79443484881 - 36561.62181134135) / 51024.79443484881 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.929110987585698 < 16.38629534960364. IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.5097404491145 (T) = (0 -14618.800972979367) / Math.Sqrt((181927.82422930378 / (299)) + (210327.7566613221 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14742337049745605 = (17146.612359652707 - 14618.800972979367) / 17146.612359652707 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Json_ToStream<IndexViewModel>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 69.36178535143623 < 95.09692787876543. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 60.10540636708805 (T) = (0 -71181.02881234557) / Math.Sqrt((32648149.00806551 / (299)) + (2894806.9498856314 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.32725324650754195 = (105806.57348819678 - 71181.02881234557) / 105806.57348819678 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22Hello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20Worldbbbbbbbbbbbbbbbbbbb.html>) 14.89 ns 12.38 ns 0.83 0.01 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22string1%22%2c%20value%3a%20%22string2%22%2c%20comparisonType%3a%20InvariantCulture).html>) 35.63 ns 29.50 ns 0.83 0.01 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22foobardzsdzs%22%2c%20value%3a%20%22rddzs%22%2c%20comparisonType%3a%20InvariantCulture).html>) 43.24 ns 37.55 ns 0.87 0.00 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22AAAAA5AAAA%22%2c%20value%3a%20%225%22%2c%20comparisonType%3a%20InvariantCulture).html>) 32.23 ns 26.18 ns 0.81 0.00 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22ABCDE%22%2c%20value%3a%20%22c%22%2c%20comparisonType%3a%20InvariantCultureIgnoreCase).html>) 31.25 ns 21.51 ns 0.69 0.01 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.378758354430735 < 14.13306964435857. IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/4/2023 7:16:27 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 6.17074027941653 (T) = (0 -12.333422605655521) / Math.Sqrt((3.7061972068914137 / (299)) + (0.00757966537961532 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.05392487884064612 = (13.036409403242429 - 12.333422605655521) / 13.036409403242429 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 29.50200755773604 < 33.877615839536745. IsChangePoint: Marked as a change because one of 3/19/2023 2:30:51 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 61.89619668864893 (T) = (0 -29.62432470919634) / Math.Sqrt((1.3270584903929974 / (299)) + (0.002474148545310415 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12445711150881537 = (33.835378139211066 - 29.62432470919634) / 33.835378139211066 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 37.54554837501381 < 41.11407023526971. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 116.34766485810886 (T) = (0 -37.54341456934383) / Math.Sqrt((0.7680826675315191 / (299)) + (0.0008032302412606389 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13714806248812397 = (43.510842286110176 - 37.54341456934383) / 43.510842286110176 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAA5AAAA", value: "5", comparisonType: InvariantCulture) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 26.180935095304132 < 30.642498664366475. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 127.90265691259654 (T) = (0 -26.193686537881135) / Math.Sqrt((0.49407297538201905 / (299)) + (0.00023208532565044803 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1663619173315796 = (31.420933235243854 - 26.193686537881135) / 31.420933235243854 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 21.50555743205364 < 29.66507321809968. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 104.78672701407544 (T) = (0 -21.499025839847867) / Math.Sqrt((1.5311658971480422 / (299)) + (0.00025379686809237596 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25895922836676355 = (29.01193384065025 - 21.499025839847867) / 29.01193384065025 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ServiceProvider - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider.html>) 35.44 ns 20.12 ns 0.57 0.18 True
[Factory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory.html>) 36.59 ns 27.96 ns 0.76 0.45 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.121303111516195 < 32.45034682957117. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 41.69447410988291 (T) = (0 -22.595397450669275) / Math.Sqrt((1.6111108459918548 / (299)) + (1.1075684657603166 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.35706955410822466 = (35.14438862718404 - 22.595397450669275) / 35.14438862718404 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.963662883403664 < 34.9980171914374. IsChangePoint: Marked as a change because one of 5/2/2023 12:51:17 PM, 5/19/2023 1:23:34 PM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 34.846816833348015 (T) = (0 -26.518276916214273) / Math.Sqrt((8.102143912829556 / (299)) + (1.7881788414255824 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.3554451688290587 = (41.142003183870955 - 26.518276916214273) / 41.142003183870955 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(TreeRecord).SerializeObjectProperty(Mode%3a%20Reflection).html>) 17.48 μs 14.76 μs 0.84 0.44 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(TreeRecord).SerializeToStream(Mode%3a%20Reflection).html>) 16.58 μs 12.95 μs 0.78 0.47 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;TreeRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.762857396030247 < 16.085044989272028. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 10.484614834536924 (T) = (0 -13792.947657643048) / Math.Sqrt((1024028.902253561 / (299)) + (1170832.9891881177 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.188589000627476 = (16998.71910574214 - 13792.947657643048) / 16998.71910574214 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<TreeRecord>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.94786073559279 < 15.96767961655753. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 6.002342603607681 (T) = (0 -14158.273380545383) / Math.Sqrt((1113373.351936237 / (299)) + (2105927.4446806535 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14718115765429987 = (16601.73612206162 - 14158.273380545383) / 16601.73612206162 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Decimal.TryParse(value%3a%20%22123456.789%22).html>) 80.51 ns 74.07 ns 0.92 0.00 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Tests.Perf_Decimal.Parse(value%3a%20%22123456.789%22).html>) 84.85 ns 77.84 ns 0.92 0.00 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Decimal.TryParse(value: "123456.789") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 74.07129778030539 < 76.88754982739803. IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 58.9250713205918 (T) = (0 -74.06585049167107) / Math.Sqrt((3.9365770155352395 / (299)) + (0.45679420040592583 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14882961775778522 = (87.01647993972891 - 74.06585049167107) / 87.01647993972891 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Decimal.Parse(value: "123456.789") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 77.83548080558307 < 80.53497994920987. IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 82.70640862219261 (T) = (0 -77.68345053224871) / Math.Sqrt((3.9979189440771234 / (299)) + (0.14931520382373073 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.14372890650460976 = (90.7229627653744 - 77.68345053224871) / 90.7229627653744 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<StructRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeObjectProperty(Mode%3a%20Reflection).html>) 773.14 ns 592.25 ns 0.77 0.21 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToString(Mode%3a%20Reflection).html>) 524.60 ns 416.19 ns 0.79 0.06 True
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 461.33 ns 377.54 ns 0.82 0.10 True
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 666.84 ns 530.41 ns 0.80 0.22 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToWriter(Mode%3a%20SourceGen).html>) 229.10 ns 194.50 ns 0.85 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 328.10 ns 285.92 ns 0.87 0.01 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToStream(Mode%3a%20Reflection).html>) 582.32 ns 504.22 ns 0.87 0.07 True
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToString(Mode%3a%20SourceGen).html>) 371.80 ns 333.31 ns 0.90 0.00 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToWriter(Mode%3a%20Reflection).html>) 380.16 ns 280.73 ns 0.74 0.08 True

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 592.2462621255991 < 749.9102134103777. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 19.497671888248377 (T) = (0 -623.8255496265666) / Math.Sqrt((628.1746720556641 / (299)) + (704.4840573704319 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.18995548019082303 = (770.1126720461264 - 623.8255496265666) / 770.1126720461264 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 416.1853978178605 < 483.7245707595346. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 69.55027260634739 (T) = (0 -414.5765384147113) / Math.Sqrt((127.23244002104316 / (299)) + (24.765024585335215 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20388942928658849 = (520.7524603563554 - 414.5765384147113) / 520.7524603563554 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 377.5350922095163 < 451.5067115262355. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 42.43843314137117 (T) = (0 -374.1636949204812) / Math.Sqrt((131.90011101989134 / (299)) + (70.61941140193277 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21561219254539954 = (477.01365493514174 - 374.1636949204812) / 477.01365493514174 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 530.4056566343593 < 597.5783998960518. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/12/2023 6:21:10 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 12.242248774556806 (T) = (0 -558.9753023872252) / Math.Sqrt((489.89577095837234 / (299)) + (557.7717580131306 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.127530300013576 = (640.6816218327388 - 558.9753023872252) / 640.6816218327388 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 194.50028045830473 < 216.79322520823726. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 70.74085114666173 (T) = (0 -193.59536485755123) / Math.Sqrt((42.5902677992584 / (299)) + (0.7193581542205004 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13978785970639984 = (225.05537388890482 - 193.59536485755123) / 225.05537388890482 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 285.9157490216163 < 309.7274056482484. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/12/2023 2:59:13 PM, 4/25/2023 6:51:20 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 76.25309354179463 (T) = (0 -285.85952199334577) / Math.Sqrt((81.03478025794614 / (299)) + (0.9466224308290166 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13525917095824289 = (330.572481826855 - 285.85952199334577) / 330.572481826855 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 504.2206534904847 < 545.5977018523776. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 24.52256786044044 (T) = (0 -473.98427010019935) / Math.Sqrt((144.54241297069652 / (299)) + (263.60097548208853 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19076358497145643 = (585.7179203724796 - 473.98427010019935) / 585.7179203724796 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 333.30823592557255 < 353.6089682945152. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 64.21588276675753 (T) = (0 -333.5878388760827) / Math.Sqrt((61.897144372349956 / (299)) + (2.8105192393038254 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1112929437102009 = (375.36310364042254 - 333.5878388760827) / 375.36310364042254 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 280.7261602907153 < 361.32977738057576. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 42.38864911415368 (T) = (0 -283.4185342834743) / Math.Sqrt((99.22362618430402 / (299)) + (79.22270603493769 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27490535839936586 = (390.871091886478 - 283.4185342834743) / 390.871091886478 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.MDBenchF.MDInvMt

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/Benchstone.MDBenchF.MDInvMt.Test.html>) 4.33 ms 3.90 ms 0.90 0.00 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchF.MDInvMt*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.MDBenchF.MDInvMt.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.8991625 < 4.110756050347222. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 197.4671304699029 (T) = (0 -3898398.3839344424) / Math.Sqrt((723847223.9743755 / (299)) + (31634385.60590287 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10039750337988011 = (4333467.7244461235 - 3898398.3839344424) / 4333467.7244461235 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 9.43 μs 7.24 μs 0.77 0.00 True
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToWriter(Mode%3a%20Reflection).html>) 9.20 μs 6.76 μs 0.73 0.00 True
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 9.42 μs 7.22 μs 0.77 0.01 True
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeObjectProperty(Mode%3a%20Reflection).html>) 10.18 μs 8.04 μs 0.79 0.02 True
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToString(Mode%3a%20SourceGen).html>) 10.01 μs 7.74 μs 0.77 0.00 True
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToWriter(Mode%3a%20SourceGen).html>) 9.17 μs 6.75 μs 0.74 0.01 True
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToStream(Mode%3a%20Reflection).html>) 9.28 μs 7.17 μs 0.77 0.00 True
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 10.29 μs 7.95 μs 0.77 0.00 True
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToStream(Mode%3a%20SourceGen).html>) 9.31 μs 7.19 μs 0.77 0.00 True
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToString(Mode%3a%20Reflection).html>) 10.03 μs 7.81 μs 0.78 0.00 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.242371031746031 < 8.960719988230268. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 128.22371615562636 (T) = (0 -7276.590509070156) / Math.Sqrt((14951.842088576723 / (299)) + (3165.602458918756 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.231888467826185 = (9473.351465609223 - 7276.590509070156) / 9473.351465609223 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.757537523091133 < 8.644051747060713. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 108.137516727375 (T) = (0 -6826.289002795424) / Math.Sqrt((14332.691920337464 / (299)) + (5347.755495303737 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2534531066609458 = (9143.818109353746 - 6826.289002795424) / 9143.818109353746 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.221008730708351 < 8.939108315247164. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 141.6862005428839 (T) = (0 -7232.602015069366) / Math.Sqrt((14023.64379558011 / (299)) + (2540.68066808078 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23369363737269444 = (9438.264338915524 - 7232.602015069366) / 9438.264338915524 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.038649889002732 < 9.696124383147879. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 71.02304857615637 (T) = (0 -8095.519092401695) / Math.Sqrt((15606.971855025922 / (299)) + (11557.366827186897 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2120737855375223 = (10274.46345077432 - 8095.519092401695) / 10274.46345077432 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.740179012128049 < 9.478471348636056. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 107.962120935722 (T) = (0 -7806.849848031274) / Math.Sqrt((11736.246100311962 / (299)) + (5145.593554662634 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22387480045234096 = (10058.7506404653 - 7806.849848031274) / 10058.7506404653 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.745542313776244 < 8.689506946863702. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 124.44739461414581 (T) = (0 -6807.503668857169) / Math.Sqrt((21315.23633986152 / (299)) + (3743.0243436339933 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25732103601078554 = (9166.145803149517 - 6807.503668857169) / 9166.145803149517 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.166162227602905 < 8.803821552763546. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 149.51061918822984 (T) = (0 -7141.834799263204) / Math.Sqrt((14846.140264076372 / (299)) + (2123.3110406751753 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2340201863081991 = (9323.789833104907 - 7141.834799263204) / 9323.789833104907 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.949884653734591 < 9.715483067797074. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 90.27231472367338 (T) = (0 -8023.884691433278) / Math.Sqrt((13098.838074682208 / (299)) + (7572.387740937418 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21969712655024798 = (10283.038758987705 - 8023.884691433278) / 10283.038758987705 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.194798983364139 < 8.860447440156742. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 157.37065016439385 (T) = (0 -7162.8315709628605) / Math.Sqrt((14721.502276109235 / (299)) + (1869.9247150728268 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.23388290329421665 = (9349.525812388503 - 7162.8315709628605) / 9349.525812388503 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.8093747797589685 < 9.503413636693848. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 119.71228333044054 (T) = (0 -7845.384488186565) / Math.Sqrt((12293.605098896702 / (299)) + (3801.550148132514 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21794129898294154 = (10031.707949778875 - 7845.384488186565) / 10031.707949778875 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 633.56 ns 548.80 ns 0.87 0.18 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToString(Mode%3a%20SourceGen).html>) 397.50 ns 364.68 ns 0.92 0.00 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 353.12 ns 314.62 ns 0.89 0.04 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToWriter(Mode%3a%20Reflection).html>) 387.03 ns 293.96 ns 0.76 0.02 True
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToStream(Mode%3a%20Reflection).html>) 616.17 ns 502.66 ns 0.82 0.02 True
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 479.30 ns 383.96 ns 0.80 0.02 True
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToWriter(Mode%3a%20SourceGen).html>) 244.00 ns 209.04 ns 0.86 0.01 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeObjectProperty(Mode%3a%20Reflection).html>) 748.27 ns 665.32 ns 0.89 0.18 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToString(Mode%3a%20Reflection).html>) 522.42 ns 431.86 ns 0.83 0.01 True

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 548.7996691670794 < 609.8108345245118. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 13.15252852237902 (T) = (0 -569.5703767056881) / Math.Sqrt((486.33004150918504 / (299)) + (384.1430760136232 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.11420967896140875 = (643.008128648172 - 569.5703767056881) / 643.008128648172 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 364.68463987855483 < 378.8001803151513. IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.99545601106069 (T) = (0 -360.0595588053897) / Math.Sqrt((70.33860092048715 / (299)) + (5.646133141082509 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.10202709024539261 = (400.9692885989006 - 360.0595588053897) / 400.9692885989006 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 314.61924291407223 < 336.23908608177817. IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 4/26/2023 5:34:01 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 51.94534417660275 (T) = (0 -315.44166337396695) / Math.Sqrt((114.19008457279293 / (299)) + (3.5534855525679965 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1176214640279712 = (357.4901819506253 - 315.44166337396695) / 357.4901819506253 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 293.95508582570574 < 367.13414633862317. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 47.96765777832926 (T) = (0 -291.0985278570111) / Math.Sqrt((57.93377807019565 / (299)) + (56.546395841186836 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.25993853323145305 = (393.3437166078689 - 291.0985278570111) / 393.3437166078689 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 502.65859783649296 < 583.0548155218522. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 49.26874146906189 (T) = (0 -478.439722973901) / Math.Sqrt((115.78767968472314 / (299)) + (76.67645507662088 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.20519719671406914 = (601.9602862444635 - 478.439722973901) / 601.9602862444635 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 383.9591023501985 < 454.7827476170473. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 72.70881030153288 (T) = (0 -387.44350423617675) / Math.Sqrt((94.22476700537796 / (299)) + (22.024849558434116 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.21012015779388696 = (490.50942122292616 - 387.44350423617675) / 490.50942122292616 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 209.0350613056894 < 229.13540099394837. IsChangePoint: Marked as a change because one of 3/20/2023 7:45:43 PM, 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 78.45520804740896 (T) = (0 -208.54087451985077) / Math.Sqrt((37.54407548780124 / (299)) + (0.3751720742788149 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.12879772634924114 = (239.37136165400906 - 208.54087451985077) / 239.37136165400906 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 665.3200133909551 < 716.4016432298703. IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 18.996920693847315 (T) = (0 -643.1470605696385) / Math.Sqrt((455.21091579820416 / (299)) + (394.3850368951273 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1428981395945972 = (750.3741273708526 - 643.1470605696385) / 750.3741273708526 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 431.8614403910548 < 495.7392680687974. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 70.49354013895406 (T) = (0 -430.48999414031016) / Math.Sqrt((94.73194845433335 / (299)) + (23.319609733180908 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.19218011033688612 = (532.9034350959568 - 430.48999414031016) / 532.9034350959568 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.BinaryWriterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteAsciiChar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.BinaryWriterTests.WriteAsciiChar.html>) 8.03 ns 6.62 ns 0.83 0.37 False
[WriteBool - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.IO.Tests.BinaryWriterTests.WriteBool.html>) 3.28 ns 0.80 ns 0.25 0.15 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.BinaryWriterTests.WriteAsciiChar ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.623388668369531 < 7.35737510820601. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 8.006557021890995 (T) = (0 -6.7598083203421595) / Math.Sqrt((0.10306034753944525 / (299)) + (0.1451225547328832 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1127361777606276 = (7.618712891145527 - 6.7598083203421595) / 7.618712891145527 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.BinaryWriterTests.WriteBool ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0.8041063194246233 < 3.116030900395418. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 126.80321189011937 (T) = (0 -0.7551306273222388) / Math.Sqrt((0.055562455586137205 / (299)) + (0.0027925596262206136 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7707016108078139 = (3.2932225559130606 - 0.7551306273222388) / 3.2932225559130606 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in BenchmarksGame.Fasta_2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[RunBench - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/BenchmarksGame.Fasta_2.RunBench.html>) 455.93 μs 318.83 μs 0.70 0.00 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.Fasta_2*'
### Payloads [Baseline]() [Compare]() ### Histogram #### BenchmarksGame.Fasta_2.RunBench ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 318.8251153846154 < 434.17049742445056. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 192.7859013106153 (T) = (0 -322184.41320924566) / Math.Sqrt((3310361.7647810993 / (299)) + (6114543.128111928 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.29337457857936067 = (455947.9512660443 - 322184.41320924566) / 455947.9512660443 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.XmlToStream(ClassImplementingIXmlSerialiable).DataContractSerializer.html>) 737.74 ns 595.43 ns 0.81 0.24 False
[DataContractSerializerBinaryXml - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/MicroBenchmarks.Serializers.Xml_ToStream(ClassImplementingIXmlSerialiable).DataContractSerializerBinaryXml.html>) 419.69 ns 281.82 ns 0.67 0.04 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;ClassImplementingIXmlSerialiable&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>.DataContractSerializer_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 595.429848423187 < 721.3828958551953. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 23.53699191839809 (T) = (0 -542.9254663508958) / Math.Sqrt((558.7955571021985 / (299)) + (776.1102573311829 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.253825200266938 = (727.6116354306297 - 542.9254663508958) / 727.6116354306297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### MicroBenchmarks.Serializers.Xml_ToStream<ClassImplementingIXmlSerialiable>.DataContractSerializer_BinaryXml_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 281.81904036518154 < 403.6852410411562. IsChangePoint: Marked as a change because one of 4/4/2023 3:37:20 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 68.2618924492807 (T) = (0 -270.95051144884707) / Math.Sqrt((1150.5265481907754 / (218)) + (21.417744468053105 / (13))) is greater than 1.9703772833260038 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (218) + (13) - 2, .975) and 0.3986714795831837 = (450.58649681381365 - 270.95051144884707) / 450.58649681381365 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Linq.Tests.Perf_OrderBy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[OrderByValueType - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_OrderBy.OrderByValueType(NumberOfPeople%3a%20512).html>) 39.31 μs 34.62 μs 0.88 0.29 False
[OrderByCustomComparer - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Linq.Tests.Perf_OrderBy.OrderByCustomComparer(NumberOfPeople%3a%20512).html>) 100.77 μs 74.90 μs 0.74 0.03 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_OrderBy*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_OrderBy.OrderByValueType(NumberOfPeople: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 34.61751659292036 < 37.364454297542785. IsChangePoint: Marked as a change because one of 2/4/2023 10:21:07 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 43.448650540418726 (T) = (0 -34726.63636680499) / Math.Sqrt((4832090.375329542 / (299)) + (14724.251500648132 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.1412871413026435 = (40440.33580617895 - 34726.63636680499) / 40440.33580617895 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Linq.Tests.Perf_OrderBy.OrderByCustomComparer(NumberOfPeople: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 74.89669413919414 < 96.78520599112427. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 157.89092046909383 (T) = (0 -74153.7664465577) / Math.Sqrt((2337843.8512600935 / (299)) + (306357.2307615019 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.27389457065241535 = (102125.34357880485 - 74153.7664465577) / 102125.34357880485 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[HeapSort - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Guid%2c%20Guid).HeapSort(Size%3a%20100).html>) 6.97 μs 5.29 μs 0.76 0.00 True
[HeapSort - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Guid%2c%20Guid).HeapSort(Size%3a%2010).html>) 287.26 ns 231.44 ns 0.81 0.00 True
[K_Max_Elements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.Tests.Perf_PriorityQueue(Guid%2c%20Guid).K_Max_Elements(Size%3a%201000).html>) 8.98 μs 6.94 μs 0.77 0.03 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 100) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.286205721186309 < 6.625602007573421. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 104.13109832132145 (T) = (0 -5228.501242884055) / Math.Sqrt((10129.349588820256 / (299)) + (3374.803452091583 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2543910776720944 = (7012.390928155568 - 5228.501242884055) / 7012.390928155568 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.HeapSort(Size: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 231.43794890473637 < 274.119017644012. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 170.89852336857174 (T) = (0 -229.5957075295237) / Math.Sqrt((14.896971048210384 / (299)) + (1.2261991085006587 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.22033494905422168 = (294.4799273111075 - 229.5957075295237) / 294.4799273111075 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.K_Max_Elements(Size: 1000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.93554682449903 < 8.18243493382278. IsChangePoint: Marked as a change because one of 4/8/2023 9:16:19 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 42.35776954617522 (T) = (0 -7165.4664119259605) / Math.Sqrt((65471.719816421355 / (299)) + (23536.28034234622 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.2103003376383989 = (9073.660219756957 - 7165.4664119259605) / 9073.660219756957 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ArrayList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Collections.IterateForNonGeneric(String).ArrayList(Size%3a%20512).html>) 1.81 μs 662.12 ns 0.37 0.01 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForNonGeneric&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForNonGeneric<String>.ArrayList(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 662.1228586039376 < 1.7005108670656832. IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM. IsImprovementStdDev: Marked as improvement because 229.00193271473717 (T) = (0 -662.5838870756377) / Math.Sqrt((6146.113221780746 / (299)) + (0.3962571335755953 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6106130694643572 = (1701.6079254744982 - 662.5838870756377) / 1701.6079254744982 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)