search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 23 Improvements on 12/6/2022 5:44:36 AM #10673

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 21a3647f09ca92cbf890129d4e917cbd01a1b374
Compare d45904acf566817af32eb20a3cbeff042c0e717d
Diff Diff

Improvements in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetFolderPath - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Environment.GetFolderPath(folder%3a%20System%2c%20option%3a%20None).html>) 539.31 ns 501.20 ns 0.93 0.10 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Environment*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Environment.GetFolderPath(folder: System, option: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 501.2001346623084 < 522.0425227734268. IsChangePoint: Marked as a change because one of 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 19.92750000181733 (T) = (0 -496.09616873618313) / Math.Sqrt((160.0898941403146 / (299)) + (283.3592226184007 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.10431570539383493 = (553.8739170974501 - 496.09616873618313) / 553.8739170974501 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 21a3647f09ca92cbf890129d4e917cbd01a1b374
Compare d45904acf566817af32eb20a3cbeff042c0e717d
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Backtracking - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options%3a%20IgnoreCase%2c%20Compiled).html>) 138.64 ns 116.20 ns 0.84 0.12 False
[OneNodeBacktracking - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options%3a%20IgnoreCase%2c%20Compiled).html>) 84.24 ns 73.07 ns 0.87 0.11 False
[Uri_IsMatch - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options%3a%20Compiled).html>) 94.65 ns 84.51 ns 0.89 0.07 False
[Uri_IsNotMatch - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 113.98 ns 98.61 ns 0.87 0.05 False
[Email_IsMatch - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options%3a%20Compiled).html>) 130.79 ns 110.50 ns 0.84 0.12 False
[Backtracking - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options%3a%20Compiled).html>) 116.46 ns 103.07 ns 0.89 0.09 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 116.19975753557586 < 130.88096948521655. IsChangePoint: Marked as a change because one of 10/26/2022 2:14:52 PM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 25.62230601599537 (T) = (0 -115.31532967734071) / Math.Sqrt((83.56814689588371 / (299)) + (6.339002958204089 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.13041392019904452 = (132.60944759342962 - 115.31532967734071) / 132.60944759342962 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 73.07191531717775 < 80.08728545068111. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 12/1/2022 5:03:18 PM, 12/5/2022 3:14:07 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 11.034479004839598 (T) = (0 -72.59340494516204) / Math.Sqrt((11.491127916422718 / (299)) + (3.6763379918239387 / (39))) is greater than 1.9670493839589689 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (39) - 2, .975) and 0.05246622568088392 = (76.61299988734079 - 72.59340494516204) / 76.61299988734079 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 84.50815943120294 < 89.49014313611583. IsChangePoint: Marked as a change because one of 11/8/2022 2:29:58 AM, 11/11/2022 5:53:56 PM, 12/5/2022 9:44:53 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 19.837766661055582 (T) = (0 -85.76866592168123) / Math.Sqrt((8.853449117100526 / (299)) + (2.519326910107226 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.06742130407430164 = (91.96936011555073 - 85.76866592168123) / 91.96936011555073 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 98.61120381210816 < 107.7908358296384. IsChangePoint: Marked as a change because one of 10/14/2022 3:33:41 AM, 10/18/2022 1:35:00 AM, 12/1/2022 7:29:59 PM, 12/7/2022 6:04:36 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.270956198723555 (T) = (0 -94.20473197401093) / Math.Sqrt((30.46602689546497 / (299)) + (5.382666052436921 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.0683479981427381 = (101.11579407999169 - 94.20473197401093) / 101.11579407999169 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 110.50106706542493 < 122.65311419286628. IsChangePoint: Marked as a change because one of 10/4/2022 2:54:32 AM, 10/4/2022 3:13:24 PM, 10/28/2022 6:19:52 PM, 11/1/2022 2:33:16 AM, 12/1/2022 7:29:59 PM, 12/7/2022 6:04:36 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 9.781728849930841 (T) = (0 -113.93335309254591) / Math.Sqrt((40.874425958646235 / (299)) + (7.776368338790143 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.054328410863661074 = (120.47877339383616 - 113.93335309254591) / 120.47877339383616 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 103.07056142192248 < 110.07775409786976. IsChangePoint: Marked as a change because one of 10/26/2022 2:14:52 PM, 11/10/2022 12:15:38 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 10.249088301916773 (T) = (0 -101.3728836301207) / Math.Sqrt((24.495543047384732 / (299)) + (22.08051053255206 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.07774821651185052 = (109.91888055418794 - 101.3728836301207) / 109.91888055418794 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 21a3647f09ca92cbf890129d4e917cbd01a1b374
Compare d45904acf566817af32eb20a3cbeff042c0e717d
Diff Diff

Improvements in System.Memory.Slice<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadOnlyMemoryStartLength - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Slice(String).ReadOnlyMemoryStartLength.html>) 6.48 ns 5.47 ns 0.84 0.22 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Slice&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Slice<String>.ReadOnlyMemoryStartLength ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 5.465730617179684 < 5.948972788185146. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/12/2022 1:08:53 AM, 10/17/2022 3:19:47 PM, 10/26/2022 7:11:30 AM, 11/18/2022 7:31:41 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 23.75391815139398 (T) = (0 -5.506587350060366) / Math.Sqrt((0.1476733427658847 / (299)) + (0.007920600990976013 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.10334704661856996 = (6.141269405620194 - 5.506587350060366) / 6.141269405620194 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 21a3647f09ca92cbf890129d4e917cbd01a1b374
Compare d45904acf566817af32eb20a3cbeff042c0e717d
Diff Diff

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseGeneric - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Enum.TryParseGeneric(text%3a%20%22Red%2c%20Orange%2c%20Yellow%2c%20Green%2c%20Blue%22).html>) 95.97 ns 82.87 ns 0.86 0.03 False
[TryParseGeneric - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Enum.TryParseGeneric(text%3a%20%22Red%22).html>) 32.72 ns 24.62 ns 0.75 0.03 False
[EnumToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Enum.EnumToString(value%3a%20Yellow).html>) 17.65 ns 15.47 ns 0.88 0.10 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Enum.TryParseGeneric(text: "Red, Orange, Yellow, Green, Blue") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 82.87319390236175 < 92.0472895389909. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 37.045677291953034 (T) = (0 -82.72493993058382) / Math.Sqrt((53.58483168370815 / (299)) + (1.25480877145397 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.17163308243931577 = (99.86509380913752 - 82.72493993058382) / 99.86509380913752 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Enum.TryParseGeneric(text: "Red") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 24.61666453257135 < 30.045294636589507. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 100.53124789297422 (T) = (0 -22.38193476305646) / Math.Sqrt((0.471063093146106 / (299)) + (0.2912628222134512 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.30632627022332914 = (32.26579557836557 - 22.38193476305646) / 32.26579557836557 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Enum.EnumToString(value: Yellow) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 15.46652993414814 < 16.865131234910383. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 44.14373073186117 (T) = (0 -14.9365838920583) / Math.Sqrt((0.2975191955353341 / (299)) + (0.05629860597825316 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.13005733205925668 = (17.16961869155694 - 14.9365838920583) / 17.16961869155694 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 21a3647f09ca92cbf890129d4e917cbd01a1b374
Compare d45904acf566817af32eb20a3cbeff042c0e717d
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22zqj%22%2c%20Options%3a%20Compiled).html>) 31.32 μs 23.08 μs 0.74 0.04 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22zqj%22%2c%20Options%3a%20None).html>) 31.29 μs 23.07 μs 0.74 0.04 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fm)%5eSherlock%20Holmes%7cSherlock%20Holmes%24%22%2c%20Options%3a%20Compiled).html>) 46.31 μs 29.55 μs 0.64 0.06 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22Sherlock%5c%5cs%2bHolmes%22%2c%20Options%3a%20None).html>) 54.54 μs 37.34 μs 0.68 0.27 False
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22aei%22%2c%20Options%3a%20Compiled).html>) 54.61 μs 43.42 μs 0.80 0.03 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 23.084457021929325 < 29.7874498997996. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/31/2022 10:23:25 PM, 11/7/2022 4:10:19 PM, 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 22.2232757745412 (T) = (0 -23076.337595085508) / Math.Sqrt((16054233.371869456 / (299)) + (3361.0321613340043 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.18256958419060687 = (28230.338813898998 - 23076.337595085508) / 28230.338813898998 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "zqj", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 23.072575086164452 < 29.77429974532398. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/27/2022 12:34:03 AM, 11/7/2022 4:10:19 PM, 11/11/2022 5:53:56 PM, 11/22/2022 2:20:42 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 19.43398565881002 (T) = (0 -23103.24665558557) / Math.Sqrt((16694700.35735948 / (299)) + (2212.3048695838415 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.16588508324049692 = (27697.918106225206 - 23103.24665558557) / 27697.918106225206 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 29.553753150598617 < 43.97223027613412. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/6/2022 2:13:23 PM, 10/11/2022 8:42:28 PM, 10/27/2022 12:34:03 AM, 11/7/2022 4:10:19 PM, 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 24.63140522174194 (T) = (0 -29484.208961620778) / Math.Sqrt((63664344.258549124 / (299)) + (5007.991000337139 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.2782988771407691 = (40853.76623055598 - 29484.208961620778) / 40853.76623055598 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\\s+Holmes", Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 37.33571214969241 < 51.37540283764369. IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 5.275131278472476 (T) = (0 -40056.69708222722) / Math.Sqrt((71428051.72503324 / (259)) + (39555942.552595824 / (36))) is greater than 1.9680934526838334 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (259) + (36) - 2, .975) and 0.13374719077785988 = (46241.34739395132 - 40056.69708222722) / 46241.34739395132 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 43.41540740740741 < 51.85547484756097. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/11/2022 8:42:28 PM, 10/18/2022 1:35:00 AM, 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 12/5/2022 9:44:53 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 29.173984933685055 (T) = (0 -43136.17171890903) / Math.Sqrt((27641123.7338499 / (299)) + (26052.079424332966 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.1711136397355751 = (52041.116595461965 - 43136.17171890903) / 52041.116595461965 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 21a3647f09ca92cbf890129d4e917cbd01a1b374
Compare d45904acf566817af32eb20a3cbeff042c0e717d
Diff Diff

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ObjectStringIsString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/PerfLabTests.LowLevelPerf.ObjectStringIsString.html>) 83.53 μs 69.64 μs 0.83 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.ObjectStringIsString ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 69.63548148148149 < 77.89518783386751. IsChangePoint: Marked as a change because one of 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 119.37006934510819 (T) = (0 -69638.24169747435) / Math.Sqrt((2968957.8293405524 / (299)) + (510.0144991324913 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.14597952492758043 = (81541.65354357475 - 69638.24169747435) / 81541.65354357475 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 21a3647f09ca92cbf890129d4e917cbd01a1b374
Compare d45904acf566817af32eb20a3cbeff042c0e717d
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22%5ba-z%5dshing%22%2c%20Options%3a%20Compiled).html>) 2.00 ms 1.82 ms 0.91 0.08 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.815179074074074 < 1.9037988160342258. IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 11/29/2022 4:49:06 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 16.496910968719945 (T) = (0 -1810990.290334023) / Math.Sqrt((10043973327.252407 / (299)) + (226819080.6231283 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.05440476548886297 = (1915185.5088084135 - 1810990.290334023) / 1915185.5088084135 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 21a3647f09ca92cbf890129d4e917cbd01a1b374
Compare d45904acf566817af32eb20a3cbeff042c0e717d
Diff Diff

Improvements in System.Threading.Tests.Perf_CancellationToken

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateLinkedTokenSource2 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Threading.Tests.Perf_CancellationToken.CreateLinkedTokenSource2.html>) 83.94 ns 71.71 ns 0.85 0.05 False
[CreateLinkedTokenSource3 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Threading.Tests.Perf_CancellationToken.CreateLinkedTokenSource3.html>) 125.34 ns 113.26 ns 0.90 0.04 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_CancellationToken*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Tests.Perf_CancellationToken.CreateLinkedTokenSource2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 71.71249301983441 < 80.12393951181664. IsChangePoint: Marked as a change because one of 11/1/2022 6:58:35 AM, 11/2/2022 9:35:40 AM, 11/16/2022 6:51:59 PM, 11/18/2022 7:31:41 AM, 12/1/2022 7:29:59 PM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 12.934912543045973 (T) = (0 -74.11108110634432) / Math.Sqrt((17.606252726436775 / (299)) + (2.4729884946327596 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.058681942950886135 = (78.7311797020776 - 74.11108110634432) / 78.7311797020776 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Threading.Tests.Perf_CancellationToken.CreateLinkedTokenSource3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 113.25921832464229 < 117.17146223246039. IsChangePoint: Marked as a change because one of 11/1/2022 6:58:35 AM, 11/2/2022 9:35:40 AM, 11/16/2022 6:51:59 PM, 11/24/2022 9:42:44 PM, 11/28/2022 10:23:06 PM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 10.866038178385935 (T) = (0 -114.38550888976013) / Math.Sqrt((100.18569367508319 / (299)) + (11.884809333234964 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.07190662811999608 = (123.24784591237162 - 114.38550888976013) / 123.24784591237162 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 21a3647f09ca92cbf890129d4e917cbd01a1b374
Compare d45904acf566817af32eb20a3cbeff042c0e717d
Diff Diff

Improvements in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.IndexerSet(String).IList(Size%3a%20512).html>) 4.40 μs 3.00 μs 0.68 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<String>.IList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 3.003700272698108 < 4.201684568308414. IsChangePoint: Marked as a change because one of 11/18/2022 8:48:12 PM, 11/22/2022 2:20:42 AM, 12/1/2022 7:29:59 PM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 7.364827886992075 (T) = (0 -3032.8840913754284) / Math.Sqrt((166757.33812604588 / (299)) + (3350.062940301441 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.05833362936408933 = (3220.7628794551842 - 3032.8840913754284) / 3220.7628794551842 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 21a3647f09ca92cbf890129d4e917cbd01a1b374
Compare d45904acf566817af32eb20a3cbeff042c0e717d
Diff Diff

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetFileNameWithoutExtension - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.IO.Tests.Perf_Path.GetFileNameWithoutExtension.html>) 30.50 ns 27.57 ns 0.90 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_Path.GetFileNameWithoutExtension ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 27.57129051632798 < 31.098367982865483. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 11/7/2022 4:10:19 PM, 11/10/2022 9:13:35 AM, 12/1/2022 7:29:59 PM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 7.657408563704936 (T) = (0 -27.123866473672063) / Math.Sqrt((2.4618024605789905 / (299)) + (1.5147908423118317 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.05955200396786325 = (28.841431517862677 - 27.123866473672063) / 28.841431517862677 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 21a3647f09ca92cbf890129d4e917cbd01a1b374
Compare d45904acf566817af32eb20a3cbeff042c0e717d
Diff Diff

Improvements in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes%3a%20100).html>) 112.62 ns 90.09 ns 0.80 0.14 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 90.09160207921106 < 104.35817481474591. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/2/2022 11:57:04 PM, 10/11/2022 8:42:28 PM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsImprovementStdDev: Marked as improvement because 5.987464027165966 (T) = (0 -99.63942256108291) / Math.Sqrt((43.85284761740988 / (299)) + (46.19693645916402 / (36))) is greater than 1.9671134448822736 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (36) - 2, .975) and 0.06703936387959628 = (106.79917105122524 - 99.63942256108291) / 106.79917105122524 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)