search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 33 Improvements on 11/18/2022 4:41:14 PM #10070

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4a223c9d50afe8ef5e00100d513312755938e05a
Compare d40dc184772e42c2da32d852c3ae5b5f5d79aa7c
Diff Diff

Improvements in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt32.TryParse(value%3a%20%2212345%22).html>) 18.61 ns 15.23 ns 0.82 0.07 False
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt32.ParseSpan(value%3a%20%220%22).html>) 11.99 ns 9.27 ns 0.77 0.05 False
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt32.ParseSpan(value%3a%20%2212345%22).html>) 16.59 ns 13.47 ns 0.81 0.08 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt32.Parse(value%3a%20%2212345%22).html>) 21.57 ns 13.40 ns 0.62 0.05 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt32.TryParse(value%3a%20%224294967295%22).html>) 23.43 ns 21.18 ns 0.90 0.12 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt32.TryParse(value%3a%20%220%22).html>) 13.11 ns 10.00 ns 0.76 0.06 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt32.Parse(value%3a%20%220%22).html>) 12.73 ns 9.26 ns 0.73 0.05 False
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt32.ParseSpan(value%3a%20%224294967295%22).html>) 22.77 ns 17.93 ns 0.79 0.10 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt32.Parse(value%3a%20%224294967295%22).html>) 25.55 ns 18.25 ns 0.71 0.11 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_UInt32.TryParse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 15.22929505318087 < 17.651649722562254. IsChangePoint: Marked as a change because one of 10/6/2022 5:26:59 PM, 10/7/2022 12:48:42 AM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 20.81986698214958 (T) = (0 -15.237098540568882) / Math.Sqrt((1.7491764367022082 / (299)) + (0.04142622435654155 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.11152500003428963 = (17.14972119773425 - 15.237098540568882) / 17.14972119773425 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.ParseSpan(value: "0") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.267367583698748 < 11.50090374008849. IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 47.530618795858636 (T) = (0 -9.40803410925192) / Math.Sqrt((0.06891311427971399 / (299)) + (0.04292112007055133 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.2096041108180262 = (11.902939068912461 - 9.40803410925192) / 11.902939068912461 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.ParseSpan(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.474804673741524 < 15.748302914744775. IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 38.574349986648045 (T) = (0 -13.486752751422092) / Math.Sqrt((0.8067020715093194 / (299)) + (0.04432951994337586 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.17241395424019676 = (16.296496081008666 - 13.486752751422092) / 16.296496081008666 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.Parse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.395349904243188 < 20.435057671309643. IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/16/2022 11:42:33 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 10.618792670759346 (T) = (0 -14.02499721204186) / Math.Sqrt((2.1346464457905157 / (299)) + (1.1936994861359682 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.15065757473699126 = (16.51277128620869 - 14.02499721204186) / 16.51277128620869 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.TryParse(value: "4294967295") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.180815526196618 < 22.98854391780126. IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/27/2022 11:22:50 AM, 9/28/2022 11:19:50 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 15.034172593525895 (T) = (0 -20.576555936439476) / Math.Sqrt((2.194834546065622 / (299)) + (0.1362398416474888 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.08425086106189379 = (22.46964267998095 - 20.576555936439476) / 22.46964267998095 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.TryParse(value: "0") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.996024935705252 < 12.792945865334834. IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 63.55039415574891 (T) = (0 -10.008396924398866) / Math.Sqrt((0.25754139198545833 / (299)) + (0.004507629543595531 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.17673737710311865 = (12.156991761852984 - 10.008396924398866) / 12.156991761852984 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.Parse(value: "0") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.262377089600603 < 11.828627826503812. IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 16.99374234399701 (T) = (0 -9.47216953899309) / Math.Sqrt((0.13716207041482745 / (299)) + (0.3479445576728853 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.20605145566748007 = (11.930457718713285 - 9.47216953899309) / 11.930457718713285 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.ParseSpan(value: "4294967295") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 17.933018760776815 < 21.307805976792437. IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 30.762494484174127 (T) = (0 -18.34313950716998) / Math.Sqrt((1.896718352928647 / (299)) + (0.059252957201020647 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.14256246862356312 = (21.392974806833916 - 18.34313950716998) / 21.392974806833916 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.Parse(value: "4294967295") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 18.246886040990336 < 24.62828532503335. IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/16/2022 6:51:59 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 12.376664331365397 (T) = (0 -18.689835295433337) / Math.Sqrt((2.367843502993088 / (299)) + (1.0138504065633247 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12916003056063693 = (21.46184827444891 - 18.689835295433337) / 21.46184827444891 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4a223c9d50afe8ef5e00100d513312755938e05a
Compare d40dc184772e42c2da32d852c3ae5b5f5d79aa7c
Diff Diff

Improvements in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt16.Parse(value%3a%20%2265535%22).html>) 16.93 ns 14.89 ns 0.88 0.06 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt16.TryParse(value%3a%20%220%22).html>) 13.25 ns 10.14 ns 0.77 0.06 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt16.TryParse(value%3a%20%2212345%22).html>) 18.50 ns 15.08 ns 0.81 0.07 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt16.Parse(value%3a%20%220%22).html>) 12.96 ns 9.77 ns 0.75 0.06 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt16.TryParse(value%3a%20%2265535%22).html>) 18.61 ns 15.16 ns 0.81 0.06 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt16.Parse(value%3a%20%2212345%22).html>) 16.80 ns 14.88 ns 0.89 0.05 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt16.ToString(value%3a%2012345).html>) 11.69 ns 8.92 ns 0.76 0.08 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_UInt16.ToString(value%3a%2065535).html>) 11.66 ns 8.97 ns 0.77 0.06 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_UInt16.Parse(value: "65535") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 14.892992683907904 < 15.868520295185393. IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 38.08345431681606 (T) = (0 -14.87247367589458) / Math.Sqrt((0.5704320291949689 / (299)) + (0.0029611311494039715 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.10486116173452346 = (16.614711640389984 - 14.87247367589458) / 16.614711640389984 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt16.TryParse(value: "0") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.14441073209444 < 12.531493799491177. IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 9/28/2022 11:19:50 AM, 11/18/2022 4:41:14 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 74.16359435483665 (T) = (0 -10.014390228787091) / Math.Sqrt((0.221152993401761 / (299)) + (0.00399865151576953 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.19022828572161427 = (12.366930151062691 - 10.014390228787091) / 12.366930151062691 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt16.TryParse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 15.075689630916242 < 17.748986186494452. IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 45.528208264705526 (T) = (0 -15.130131186343446) / Math.Sqrt((0.7399794076065185 / (299)) + (0.0020182447594568613 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.1328816587419555 = (17.44875003380985 - 15.130131186343446) / 17.44875003380985 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt16.Parse(value: "0") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 9.772535316622463 < 12.588780198587616. IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/18/2022 1:00:00 AM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 13.660093840639478 (T) = (0 -9.992089975421184) / Math.Sqrt((0.812769180782018 / (299)) + (0.27466121929271764 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.16162055457603317 = (11.918338444435745 - 9.992089975421184) / 11.918338444435745 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt16.TryParse(value: "65535") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 15.157512512898078 < 17.714448308920808. IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 41.719907139864105 (T) = (0 -15.150865333306362) / Math.Sqrt((0.7366721538299202 / (299)) + (0.00957663303360031 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.13156960554435976 = (17.446263316017866 - 15.150865333306362) / 17.446263316017866 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt16.Parse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 14.882860939746232 < 15.899865001197124. IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 38.71846977510843 (T) = (0 -14.861400938160646) / Math.Sqrt((0.5293495992036572 / (299)) + (0.004642364712671646 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.10575617390156548 = (16.61895839191935 - 14.861400938160646) / 16.61895839191935 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt16.ToString(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.919796509899784 < 11.09419770470418. IsChangePoint: Marked as a change because one of 10/4/2022 3:13:24 PM, 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 21.806283277667557 (T) = (0 -8.958963041916947) / Math.Sqrt((1.940895095964669 / (299)) + (0.005418172271335525 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.16726478527172806 = (10.758477464941034 - 8.958963041916947) / 10.758477464941034 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt16.ToString(value: 65535) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.972475014701129 < 11.079920138702564. IsChangePoint: Marked as a change because one of 10/4/2022 3:13:24 PM, 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 22.097042216239736 (T) = (0 -8.961471433604043) / Math.Sqrt((1.913926244646633 / (299)) + (0.0026312775867680972 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.16642377786541515 = (10.750632270502997 - 8.961471433604043) / 10.750632270502997 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4a223c9d50afe8ef5e00100d513312755938e05a
Compare d40dc184772e42c2da32d852c3ae5b5f5d79aa7c
Diff Diff

Improvements in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int16.TryParse(value%3a%20%220%22).html>) 13.37 ns 11.10 ns 0.83 0.06 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int16.TryParse(value%3a%20%22-32768%22).html>) 17.70 ns 15.09 ns 0.85 0.06 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int16.ToString(value%3a%20-32768).html>) 22.98 ns 17.06 ns 0.74 0.03 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int16.ToString(value%3a%2032767).html>) 13.06 ns 8.97 ns 0.69 0.06 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int16.Parse(value%3a%20%2232767%22).html>) 16.99 ns 13.41 ns 0.79 0.04 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int16.Parse(value%3a%20%22-32768%22).html>) 16.75 ns 13.91 ns 0.83 0.05 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int16.TryParse(value%3a%20%2232767%22).html>) 18.00 ns 14.57 ns 0.81 0.05 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int16.Parse(value%3a%20%220%22).html>) 13.38 ns 10.48 ns 0.78 0.06 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int16.ToString(value%3a%200).html>) 3.40 ns 2.23 ns 0.65 0.13 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int16.TryParse(value: "0") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.103033272644803 < 12.674481124424053. IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 16.941838153379106 (T) = (0 -11.189097347564033) / Math.Sqrt((0.14521994049671777 / (299)) + (0.1572284682343396 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.13326950654223105 = (12.909546199217942 - 11.189097347564033) / 12.909546199217942 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int16.TryParse(value: "-32768") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 15.086299102657772 < 16.631038600880196. IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 42.059993283767795 (T) = (0 -15.095447310561275) / Math.Sqrt((0.3953557522379954 / (299)) + (0.00110355492448359 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.09414194701412958 = (16.66425248503778 - 15.095447310561275) / 16.66425248503778 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int16.ToString(value: -32768) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 17.06379364425822 < 21.82543259718064. IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 9/28/2022 11:19:50 AM, 10/11/2022 2:16:14 AM, 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 60.2857494837324 (T) = (0 -17.09407751959678) / Math.Sqrt((1.9123239959630256 / (299)) + (0.018224029169996374 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.23357777192718776 = (22.30373401692728 - 17.09407751959678) / 22.30373401692728 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int16.ToString(value: 32767) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.970716084134335 < 12.401517425325077. IsChangePoint: Marked as a change because one of 10/4/2022 2:54:32 AM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 59.84638464642762 (T) = (0 -8.990885663934577) / Math.Sqrt((1.7444978487732874 / (299)) + (0.008406212084232579 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.34620950571874004 = (13.751936962342416 - 8.990885663934577) / 13.751936962342416 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int16.Parse(value: "32767") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.411487282770723 < 16.05978119248718. IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 81.36913400672543 (T) = (0 -13.472692120293342) / Math.Sqrt((0.19124649425045048 / (299)) + (0.006916902686970628 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.16344410727896222 = (16.104951548988748 - 13.472692120293342) / 16.104951548988748 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int16.Parse(value: "-32768") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.906030995855145 < 16.164947479929424. IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 28.321518426745005 (T) = (0 -14.037717167423766) / Math.Sqrt((0.36062260534098584 / (299)) + (0.05841023105096853 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.1208471982667951 = (15.967323472949325 - 14.037717167423766) / 15.967323472949325 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int16.TryParse(value: "32767") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 14.567249237219524 < 17.09045074875626. IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 66.58019412161477 (T) = (0 -14.560334872033218) / Math.Sqrt((0.33944225211145973 / (299)) + (0.0005111628442053234 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.13511560741439577 = (16.835007079390753 - 14.560334872033218) / 16.835007079390753 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int16.Parse(value: "0") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 10.480121818310542 < 12.365840056575372. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 49.53615600674226 (T) = (0 -10.00467965085221) / Math.Sqrt((0.19346093098428857 / (299)) + (0.021875198014407727 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.17880224585720952 = (12.183033380669219 - 10.00467965085221) / 12.183033380669219 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int16.ToString(value: 0) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.2260942653432174 < 3.23329382666458. IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 39.85682028944202 (T) = (0 -2.2027496829381343) / Math.Sqrt((0.044100093404910765 / (299)) + (0.0032596692503392235 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.24996135807880515 = (2.93684826330531 - 2.2027496829381343) / 2.93684826330531 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4a223c9d50afe8ef5e00100d513312755938e05a
Compare d40dc184772e42c2da32d852c3ae5b5f5d79aa7c
Diff Diff

Improvements in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_SByte.TryParse(value%3a%20%22127%22).html>) 16.26 ns 12.86 ns 0.79 0.06 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_SByte.Parse(value%3a%20%22127%22).html>) 14.45 ns 11.80 ns 0.82 0.05 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_SByte.ToString(value%3a%20-128).html>) 23.82 ns 15.99 ns 0.67 0.03 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_SByte.TryParse(value%3a%20%22-128%22).html>) 16.10 ns 13.36 ns 0.83 0.06 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_SByte.ToString(value%3a%20127).html>) 12.13 ns 8.15 ns 0.67 0.04 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_SByte.Parse(value%3a%20%22-128%22).html>) 14.49 ns 12.26 ns 0.85 0.05 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_SByte.TryParse(value: "127") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 12.861986880521568 < 15.42407370490786. IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 33.364023576445796 (T) = (0 -12.925471751345636) / Math.Sqrt((0.3683451810419045 / (299)) + (0.04138506506158921 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.13756398379236812 = (14.987166013987318 - 12.925471751345636) / 14.987166013987318 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_SByte.Parse(value: "127") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.803580392479274 < 13.67150167459585. IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/28/2022 6:40:13 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 82.21912002097913 (T) = (0 -11.838497647477801) / Math.Sqrt((0.23252604824491005 / (299)) + (0.0024620337893115863 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.17419577083614665 = (14.335719325952793 - 11.838497647477801) / 14.335719325952793 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_SByte.ToString(value: -128) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 15.987013228519702 < 22.628958680060784. IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 9/28/2022 11:19:50 AM, 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 53.561151825342364 (T) = (0 -16.03311940498834) / Math.Sqrt((1.959390574562202 / (299)) + (0.024761780074324644 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.23016456951909448 = (20.82668421090034 - 16.03311940498834) / 20.82668421090034 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_SByte.TryParse(value: "-128") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.364988889168247 < 15.094492878705713. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/11/2022 5:53:56 PM, 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 44.86741108900779 (T) = (0 -13.370594046401337) / Math.Sqrt((0.337506607056755 / (299)) + (0.0005085548742067848 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.10259036571834731 = (14.899097954418624 - 13.370594046401337) / 14.899097954418624 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_SByte.ToString(value: 127) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 8.152322486151686 < 11.524555860167384. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 100.78360202245078 (T) = (0 -8.121772938888917) / Math.Sqrt((0.37537541875964836 / (299)) + (0.0025786867064933722 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.317631782635341 = (11.902331808851853 - 8.121772938888917) / 11.902331808851853 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_SByte.Parse(value: "-128") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 12.257969953893836 < 13.75714417018914. IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/29/2022 7:42:19 AM, 10/25/2022 2:34:53 PM, 11/18/2022 7:31:41 AM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 47.58749527655152 (T) = (0 -12.433768551109049) / Math.Sqrt((0.19240826555873874 / (299)) + (0.017507646081406866 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.1353682741126715 = (14.380421373445314 - 12.433768551109049) / 14.380421373445314 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 4a223c9d50afe8ef5e00100d513312755938e05a
Compare d40dc184772e42c2da32d852c3ae5b5f5d79aa7c
Diff Diff

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).GetHashCodeBenchmark.html>) 13.15 ns 12.07 ns 0.92 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 12.065897835975708 < 12.513522253592432. IsChangePoint: Marked as a change because one of 11/18/2022 3:21:53 PM, 11/22/2022 2:20:42 AM falls between 11/13/2022 1:28:42 PM and 11/22/2022 2:20:42 AM. IsImprovementStdDev: Marked as improvement because 36.931369947824905 (T) = (0 -12.07652105190218) / Math.Sqrt((0.04431219335624039 / (43)) + (0.00034414251391992905 / (16))) is greater than 2.002465459290109 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .975) and 0.0902389344526721 = (13.27438764884573 - 12.07652105190218) / 13.27438764884573 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)