search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 33 Regressions on 4/10/2023 4:31:44 PM #16616

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 58ea9cbf7a342bd57a88b7fd12d4e1e7965c0889
Compare f516d02550e5f47d12b7d551b8b3dfcf33d8743a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Join_List - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.Join_List.html>) 55.24 ns 69.05 ns 1.25 0.01 False
[ToUpper - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.ToUpper(s%3a%20%22test%22).html>) 26.75 ns 30.02 ns 1.12 0.04 False
[Trim - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.Trim(s%3a%20%22Test%20%22).html>) 15.34 ns 16.80 ns 1.10 0.04 False
[Concat_str_str - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.Concat_str_str(size%3a%20100).html>) 27.55 ns 31.10 ns 1.13 0.05 False
[Trim - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.Trim(s%3a%20%22%20Te%20st%20%20%22).html>) 16.50 ns 17.68 ns 1.07 0.05 False
[Trim - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.Trim(s%3a%20%22%20Test%22).html>) 14.52 ns 15.64 ns 1.08 0.04 False
[ToUpper - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.ToUpper(s%3a%20%22TeSt%22).html>) 26.77 ns 29.39 ns 1.10 0.03 False
[ToUpperInvariant - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.ToUpperInvariant(s%3a%20%22TeSt%22).html>) 18.69 ns 20.76 ns 1.11 0.04 False
[TrimStart - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.TrimStart(s%3a%20%22%20Test%22).html>) 11.94 ns 13.05 ns 1.09 0.04 False
[CtorCharCount - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.CtorCharCount(size%3a%20100).html>) 15.50 ns 17.00 ns 1.10 0.06 False
[Concat_str_str_str - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.Concat_str_str_str(size%3a%20100).html>) 38.29 ns 43.68 ns 1.14 0.03 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Join_List ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 69.0487234726188 > 58.05122369232668. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -64.74629250871544 (T) = (0 -69.87830558764372) / Math.Sqrt((0.06992698520526283 / (21)) + (0.7452658650699688 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.261027613432646 = (55.41377908246421 - 69.87830558764372) / 55.41377908246421 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_String.ToUpper(s: "test") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 30.022456178906765 > 28.246896703041195. IsChangePoint: Marked as a change because one of 2/6/2023 9:02:07 PM, 2/9/2023 12:40:40 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -12.848105478553723 (T) = (0 -30.913097327814576) / Math.Sqrt((0.27390662275805205 / (21)) + (1.2468249182920592 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.1433222736082818 = (27.037956000152093 - 30.913097327814576) / 27.037956000152093 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_String.Trim(s: "Test ") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.80410237294641 > 16.136872409870364. IsChangePoint: Marked as a change because one of 2/4/2023 4:58:40 AM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/21/2023 10:36:00 PM, 4/4/2023 4:23:54 PM, 4/9/2023 11:04:55 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -19.08123462389856 (T) = (0 -17.771526090836513) / Math.Sqrt((0.0014565984742252248 / (20)) + (0.267053125464495 / (17))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (17) - 2, .025) and -0.15591438980219893 = (15.374431054429206 - 17.771526090836513) / 15.374431054429206 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_String.Concat_str_str(size: 100) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.09961272718406 > 28.832201793375074. IsChangePoint: Marked as a change because one of 2/4/2023 4:58:40 AM, 4/6/2023 9:09:21 PM, 4/11/2023 3:40:00 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -11.498208415323358 (T) = (0 -30.623661087136817) / Math.Sqrt((0.34388132499795476 / (27)) + (0.4587398859093386 / (10))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (10) - 2, .025) and -0.09998805835411191 = (27.839994129534762 - 30.623661087136817) / 27.839994129534762 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_String.Trim(s: " Te st ") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.68013841042621 > 17.32358617807208. IsChangePoint: Marked as a change because one of 2/4/2023 4:58:40 AM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/23/2023 6:11:55 PM, 4/4/2023 8:16:00 PM, 4/5/2023 6:15:03 PM, 4/9/2023 11:04:55 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -15.056972258650903 (T) = (0 -18.90570364162275) / Math.Sqrt((0.0013038801923757675 / (20)) + (0.4405622529631318 / (17))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (17) - 2, .025) and -0.14727813739653758 = (16.478744800737285 - 18.90570364162275) / 16.478744800737285 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_String.Trim(s: " Test") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.637731663641961 > 15.176291974093122. IsChangePoint: Marked as a change because one of 2/4/2023 4:58:40 AM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/23/2023 6:11:55 PM, 4/9/2023 11:04:55 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -15.698225217666828 (T) = (0 -16.883583142447186) / Math.Sqrt((0.0025835274043918867 / (20)) + (0.38879351186262645 / (16))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (16) - 2, .025) and -0.170033749471084 = (14.429996698881073 - 16.883583142447186) / 14.429996698881073 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_String.ToUpper(s: "TeSt") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.390764258007387 > 28.10363097612209. IsChangePoint: Marked as a change because one of 2/6/2023 9:02:07 PM, 2/9/2023 12:40:40 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -12.758544723672877 (T) = (0 -30.988862008933648) / Math.Sqrt((0.25911325326880064 / (21)) + (1.2748400194385965 / (15))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (15) - 2, .025) and -0.14737348642450376 = (27.008521963935664 - 30.988862008933648) / 27.008521963935664 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_String.ToUpperInvariant(s: "TeSt") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.76156879685373 > 19.611227586142846. IsChangePoint: Marked as a change because one of 2/4/2023 4:58:40 AM, 2/23/2023 8:54:44 PM, 3/6/2023 11:14:52 AM, 4/9/2023 11:04:55 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -20.688188290796496 (T) = (0 -21.50731079371066) / Math.Sqrt((0.014004675985606098 / (20)) + (0.28628112741386796 / (17))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (17) - 2, .025) and -0.14599484574898344 = (18.7673713136591 - 21.50731079371066) / 18.7673713136591 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_String.TrimStart(s: " Test") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.045115866013155 > 12.517555226327355. IsChangePoint: Marked as a change because one of 2/4/2023 4:58:40 AM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/23/2023 6:11:55 PM, 3/31/2023 8:05:26 PM, 4/4/2023 4:23:54 PM, 4/5/2023 6:15:03 PM, 4/9/2023 11:04:55 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -13.497366375123484 (T) = (0 -14.306321044717427) / Math.Sqrt((0.0009681317545834816 / (20)) + (0.5292264406269848 / (17))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (17) - 2, .025) and -0.19989283277664882 = (11.922998999511853 - 14.306321044717427) / 11.922998999511853 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_String.CtorCharCount(size: 100) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.00076576645145 > 16.192667311617566. IsChangePoint: Marked as a change because one of 2/4/2023 4:58:40 AM, 2/20/2023 10:31:48 AM, 3/15/2023 5:33:41 AM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -13.437801262618665 (T) = (0 -17.675562646740907) / Math.Sqrt((0.09952341970253245 / (21)) + (0.3138158184021477 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.13460922662691466 = (15.578546544424528 - 17.675562646740907) / 15.578546544424528 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_String.Concat_str_str_str(size: 100) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 43.678065972795864 > 40.088374380544266. IsChangePoint: Marked as a change because one of 3/3/2023 4:59:02 PM, 3/10/2023 8:06:53 PM, 4/6/2023 9:09:21 PM, 4/11/2023 3:40:00 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -12.238787955188904 (T) = (0 -42.611120678229994) / Math.Sqrt((0.5959362223770991 / (27)) + (1.0735009356167229 / (10))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (10) - 2, .025) and -0.11523534493348249 = (38.20818706276881 - 42.611120678229994) / 38.20818706276881 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 58ea9cbf7a342bd57a88b7fd12d4e1e7965c0889
Compare f516d02550e5f47d12b7d551b8b3dfcf33d8743a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Ctor - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern%3a%20%22%5b%5c%5cw%5c%5c.%2b-%5d%2b%40%5b%5c%5cw%5c%5c.-%5d%2b%5c%5c.%5b%5c%5cw%5c%5c.-%5d%2b%22%2c%20Options%3a%20Compiled).html>) 19.63 μs 21.16 μs 1.08 0.00 False
[Ctor - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern%3a%20%22(%3f%3a(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%5c%5c.)%7b3%7d(%3f%3a25%5b0-5%5d%7c2%5b0-4%5d%5b0-9%5d%7c%5b01%5d%3f%5b0-9%5d%5b0-9%5d)%22%2c%20Options%3a%20Compiled).html>) 29.65 μs 31.79 μs 1.07 0.01 False
[Ctor - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern%3a%20%22%5b%5c%5cw%5d%2b%3a%2f%2f%5b%5e%2f%5c%5cs%3f%23%5d%2b%5b%5e%5c%5cs%3f%23%5d%2b(%3f%3a%5c%5c%3f%5b%5e%5c%5cs%23%5d)%3f(%3f%3a%23%5b%5e%5c%5cs%5d)%3f%22%2c%20Options%3a%20Compiled).html>) 26.09 μs 29.32 μs 1.12 0.02 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "[\\w\\.+-]+@[\\w\\.-]+\\.[\\w\\.-]+", Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.160040075635706 > 20.482458181267155. IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -19.186263180757873 (T) = (0 -20940.483891262247) / Math.Sqrt((16027.109377440884 / (21)) + (62024.67936584355 / (15))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (15) - 2, .025) and -0.06851765733183271 = (19597.695693258058 - 20940.483891262247) / 19597.695693258058 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.788893816523053 > 31.142534694300245. IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -21.928449451406376 (T) = (0 -31998.856871264197) / Math.Sqrt((51222.73261923803 / (21)) + (135404.42909400008 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.07706698287392351 = (29709.25428043674 - 31998.856871264197) / 29709.25428043674 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Ctor(Pattern: "[\\w]+://[^/\\s?#]+[^\\s?#]+(?:\\?[^\\s#]*)?(?:#[^\\s]*)?", Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.318092341288413 > 27.583481187692136. IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -26.389116631778098 (T) = (0 -28973.847200206154) / Math.Sqrt((68598.32916947108 / (21)) + (117902.44882883511 / (15))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (15) - 2, .025) and -0.1062844331414104 = (26190.233119281886 - 28973.847200206154) / 26190.233119281886 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 58ea9cbf7a342bd57a88b7fd12d4e1e7965c0889
Compare f516d02550e5f47d12b7d551b8b3dfcf33d8743a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MatchWord - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options%3a%20None).html>) 1.62 μs 1.75 μs 1.08 0.19 False
[Ctor - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options%3a%20IgnoreCase%2c%20Compiled).html>) 30.69 μs 32.44 μs 1.06 0.01 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7458033828916182 > 1.689465427713799. IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/20/2023 10:31:48 AM, 3/25/2023 12:18:49 AM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -19.4366726052209 (T) = (0 -1737.928135446192) / Math.Sqrt((85.13049989070404 / (20)) + (640.8354199446601 / (16))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (16) - 2, .025) and -0.08043267431385573 = (1608.5482943672434 - 1737.928135446192) / 1608.5482943672434 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 32.44137004300048 > 32.147052539911876. IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -41.067042276545685 (T) = (0 -32507.931036453778) / Math.Sqrt((18412.412226641904 / (21)) + (21776.108305570033 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.06355870841678939 = (30565.243628953023 - 32507.931036453778) / 30565.243628953023 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 58ea9cbf7a342bd57a88b7fd12d4e1e7965c0889
Compare f516d02550e5f47d12b7d551b8b3dfcf33d8743a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.CtorFromCollection(String).Dictionary(Size%3a%20512).html>) 3.77 μs 4.19 μs 1.11 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.193919173904052 > 3.976508690286693. IsChangePoint: Marked as a change because one of 2/15/2023 2:23:18 PM, 3/3/2023 11:43:56 AM, 3/25/2023 12:18:49 AM, 3/31/2023 8:05:26 PM, 4/6/2023 9:09:21 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -14.615313335226967 (T) = (0 -4133.708103208149) / Math.Sqrt((2317.972561124739 / (21)) + (5890.081987385307 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.08382486013270636 = (3814.0000799594195 - 4133.708103208149) / 3814.0000799594195 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 58ea9cbf7a342bd57a88b7fd12d4e1e7965c0889
Compare f516d02550e5f47d12b7d551b8b3dfcf33d8743a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReflectBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Numerics.Tests.Perf_Vector3.ReflectBenchmark.html>) 1.10 ns 3.73 ns 3.40 0.19 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Vector3.ReflectBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.729330086833541 > 1.151096032348127. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/8/2023 9:36:36 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -7.414790958304701 (T) = (0 -3.099336417067297) / Math.Sqrt((2.5811187629714636E-07 / (20)) + (1.167331896932745 / (16))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (16) - 2, .025) and -1.826461118270917 = (1.0965430930687314 - 3.099336417067297) / 1.0965430930687314 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 58ea9cbf7a342bd57a88b7fd12d4e1e7965c0889
Compare f516d02550e5f47d12b7d551b8b3dfcf33d8743a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetCustomAttributesMethodBaseMissInherit - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Reflection.Attributes.GetCustomAttributesMethodBaseMissInherit.html>) 120.90 ns 144.78 ns 1.20 0.10 False
[GetCustomAttributesClassMiss - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Reflection.Attributes.GetCustomAttributesClassMiss.html>) 93.62 ns 110.29 ns 1.18 0.09 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Reflection.Attributes.GetCustomAttributesMethodBaseMissInherit ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 144.7830448829775 > 126.25788544988079. IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/13/2023 6:54:14 PM, 3/8/2023 9:36:36 PM, 4/4/2023 4:23:54 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -13.68866872241053 (T) = (0 -141.60200655132775) / Math.Sqrt((35.85276792870239 / (21)) + (4.102346246830466 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.15669079417280155 = (122.41993042971637 - 141.60200655132775) / 122.41993042971637 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Reflection.Attributes.GetCustomAttributesClassMiss ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 110.29463302841667 > 99.18841743233627. IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/8/2023 9:36:36 PM, 3/17/2023 10:06:19 AM, 3/31/2023 8:05:26 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -21.925841315245282 (T) = (0 -115.11259855157843) / Math.Sqrt((1.3927621150311262 / (21)) + (11.121081569224163 / (15))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (15) - 2, .025) and -0.20654193948788166 = (95.40704287530869 - 115.11259855157843) / 95.40704287530869 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 58ea9cbf7a342bd57a88b7fd12d4e1e7965c0889
Compare f516d02550e5f47d12b7d551b8b3dfcf33d8743a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WritePartialCharArray - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.IO.Tests.Perf_StreamWriter.WritePartialCharArray(writeLength%3a%20100).html>) 356.73 ms 389.16 ms 1.09 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_StreamWriter*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_StreamWriter.WritePartialCharArray(writeLength: 100) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 389.16023333333334 > 374.7799580000001. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/20/2023 10:31:48 AM, 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -76.20468102513243 (T) = (0 -388950344.7916667) / Math.Sqrt((2697170304210.8 / (21)) + (747296556272.8661 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.08931782501693784 = (357058643.36298627 - 388950344.7916667) / 357058643.36298627 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 58ea9cbf7a342bd57a88b7fd12d4e1e7965c0889
Compare f516d02550e5f47d12b7d551b8b3dfcf33d8743a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsFalse(String).Queue(Size%3a%20512).html>) 824.64 μs 886.59 μs 1.08 0.01 True
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsFalse(String).Stack(Size%3a%20512).html>) 833.36 μs 889.99 μs 1.07 0.01 True
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsFalse(String).Span(Size%3a%20512).html>) 712.77 μs 886.72 μs 1.24 0.18 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsFalse(String).List(Size%3a%20512).html>) 828.93 μs 896.38 μs 1.08 0.00 True
[ICollection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsFalse(String).ICollection(Size%3a%20512).html>) 829.75 μs 888.93 μs 1.07 0.00 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 886.591484375 > 872.5410005482455. IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -62.846214239079444 (T) = (0 -889980.8356723933) / Math.Sqrt((9499033.703890095 / (21)) + (7265440.156180958 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.07207587350228666 = (830147.2476616601 - 889980.8356723933) / 830147.2476616601 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Collections.ContainsFalse<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 889.9857390873016 > 874.790322368421. IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -10.30114584483883 (T) = (0 -900632.9361025261) / Math.Sqrt((8542666.15867736 / (21)) + (679179889.2517362 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.08093588400862972 = (833197.3703773682 - 900632.9361025261) / 833197.3703773682 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Collections.ContainsFalse<String>.Span(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 886.7196217105262 > 749.1475984120046. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -2.4262699956058547 (T) = (0 -794591.5681722334) / Math.Sqrt((1712124047.5088258 / (32)) + (2652396329.723056 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.07966832592853072 = (735958.9506239084 - 794591.5681722334) / 735958.9506239084 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Collections.ContainsFalse<String>.List(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 896.382039835165 > 871.925493326164. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -65.54588699861999 (T) = (0 -891038.4333139372) / Math.Sqrt((3049791.752057365 / (21)) + (11448817.48781435 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.07324785970625869 = (830226.1451122846 - 891038.4333139372) / 830226.1451122846 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Collections.ContainsFalse<String>.ICollection(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 888.9262286324786 > 871.6055715460526. IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -72.12099591010067 (T) = (0 -889981.7738401284) / Math.Sqrt((1446548.8869973843 / (21)) + (9945516.728285734 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.07219895804977906 = (830052.8247657644 - 889981.7738401284) / 830052.8247657644 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 58ea9cbf7a342bd57a88b7fd12d4e1e7965c0889
Compare f516d02550e5f47d12b7d551b8b3dfcf33d8743a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22AAAAA5AAAA%22%2c%20value%3a%20%225%22%2c%20comparisonType%3a%20InvariantCulture).html>) 26.62 ns 30.42 ns 1.14 0.04 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22foobardzsdzs%22%2c%20value%3a%20%22rddzs%22%2c%20comparisonType%3a%20InvariantCulture).html>) 35.77 ns 38.48 ns 1.08 0.06 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22ABCDE%22%2c%20value%3a%20%22c%22%2c%20comparisonType%3a%20InvariantCultureIgnoreCase).html>) 23.42 ns 28.18 ns 1.20 0.01 True
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22string1%22%2c%20value%3a%20%22string2%22%2c%20comparisonType%3a%20InvariantCulture).html>) 27.46 ns 29.83 ns 1.09 0.05 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAA5AAAA", value: "5", comparisonType: InvariantCulture) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 30.417437336151785 > 27.99639237393276. IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 3/6/2023 11:14:52 AM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -73.35981337789003 (T) = (0 -30.375427552175573) / Math.Sqrt((0.005466824450441831 / (21)) + (0.036590585040280676 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.1388103545332915 = (26.672946405219562 - 30.375427552175573) / 26.672946405219562 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 38.481957644080374 > 37.55981585696573. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 2/20/2023 10:31:48 AM, 3/25/2023 12:18:49 AM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -19.451430716014425 (T) = (0 -39.12659385076054) / Math.Sqrt((0.019122576034846857 / (21)) + (0.45850379685319126 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.09347433072076478 = (35.78190429488199 - 39.12659385076054) / 35.78190429488199 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.18314231946348 > 24.578116148273917. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -72.58218704675201 (T) = (0 -27.871549651638166) / Math.Sqrt((0.0002477838223869255 / (21)) + (0.06040319042768516 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.1908402691890229 = (23.404943864233815 - 27.871549651638166) / 23.404943864233815 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.834230490596102 > 28.867770927339784. IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 3/25/2023 12:18:49 AM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -13.941972438682328 (T) = (0 -30.936846062092663) / Math.Sqrt((0.002154588381726605 / (21)) + (0.9820252293401175 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.1257976542740356 = (27.479934733068987 - 30.936846062092663) / 27.479934733068987 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 58ea9cbf7a342bd57a88b7fd12d4e1e7965c0889
Compare f516d02550e5f47d12b7d551b8b3dfcf33d8743a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsTrue(String).ImmutableArray(Size%3a%20512).html>) 420.79 μs 470.13 μs 1.12 0.01 False
[ICollection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsTrue(String).ICollection(Size%3a%20512).html>) 415.92 μs 446.36 μs 1.07 0.05 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 470.1326715686275 > 442.08281433365926. IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/24/2023 9:01:57 AM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -12.671187997481924 (T) = (0 -464767.52337705606) / Math.Sqrt((435049.47364170186 / (21)) + (190978334.18491274 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.10408620195866757 = (420952.2069495576 - 464767.52337705606) / 420952.2069495576 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Collections.ContainsTrue<String>.ICollection(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 446.3566773504274 > 437.06549835526323. IsChangePoint: Marked as a change because one of 3/29/2023 12:02:44 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -14.263646383783335 (T) = (0 -449265.38188894436) / Math.Sqrt((82520900.96439455 / (32)) + (8020589.965940107 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.06944219918997893 = (420093.1871112143 - 449265.38188894436) / 420093.1871112143 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 58ea9cbf7a342bd57a88b7fd12d4e1e7965c0889
Compare f516d02550e5f47d12b7d551b8b3dfcf33d8743a
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Char_ToLowerInvariant - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Char.Char_ToLowerInvariant(input%3a%20%22Hello%20World!%22).html>) 11.90 ns 14.36 ns 1.21 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.35731997433764 > 12.50125423400063. IsChangePoint: Marked as a change because one of 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -37.56568372443656 (T) = (0 -14.816632024703166) / Math.Sqrt((1.563542999387444E-05 / (21)) + (0.09603502243648727 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.24445768924234834 = (11.90609544445327 - 14.816632024703166) / 11.90609544445327 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)