search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 4/26/2022 11:56:10 PM #5093

Closed performanceautofiler[bot] closed 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline e4163ea55ebb3673c29e1c2a850a6a790029d278
Compare 4ed596ef63e60ce54cfb41d55928f0fe45f65cf3
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Delete - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.Delete.html>) 8.26 μs 4.34 μs 0.53 0.73 False
[AppendAllText - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.AppendAllText(size%3a%20100).html>) 6.61 μs 5.46 μs 0.83 0.07 True
[AppendAllLinesAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.AppendAllLinesAsync.html>) 15.44 μs 13.70 μs 0.89 0.18 False
[WriteAllTextAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.WriteAllTextAsync(size%3a%2010000).html>) 29.97 μs 24.03 μs 0.80 0.07 True
[WriteAllText - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.WriteAllText(size%3a%20100000).html>) 72.96 μs 66.32 μs 0.91 0.01 False
[CopyToOverwrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.CopyToOverwrite(size%3a%204096).html>) 47.05 μs 42.93 μs 0.91 0.10 False
[CopyTo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.CopyTo(size%3a%204096).html>) 27.53 μs 21.72 μs 0.79 0.06 True
[WriteAllBytesAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.WriteAllBytesAsync(size%3a%20512).html>) 42.79 μs 38.60 μs 0.90 0.36 False
[AppendAllLines - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.AppendAllLines.html>) 9.32 μs 7.47 μs 0.80 0.10 True
[AppendAllTextAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.AppendAllTextAsync(size%3a%2010000).html>) 18.54 μs 15.76 μs 0.85 0.17 False
[WriteAllBytesAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.WriteAllBytesAsync(size%3a%204096).html>) 42.86 μs 38.55 μs 0.90 0.39 False
[WriteAllText - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.WriteAllText(size%3a%2010000).html>) 26.94 μs 21.25 μs 0.79 0.05 True
[AppendAllTextAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.AppendAllTextAsync(size%3a%20100).html>) 9.94 μs 7.88 μs 0.79 0.38 False
[AppendAllText - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.AppendAllText(size%3a%2010000).html>) 13.73 μs 11.21 μs 0.82 0.06 True
[CopyTo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_File.CopyTo(size%3a%20512).html>) 27.42 μs 21.43 μs 0.78 0.05 True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_File.Delete ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 4.337586666666666 < 7.706092. IsChangePoint: Marked as a change because one of 4/26/2022 9:50:05 PM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 12.281069599842176 (T) = (0 -5247.200223214286) / Math.Sqrt((1240952.860414621 / (299)) + (898321.3119550539 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.29584250601599793 = (7451.742355998413 - 5247.200223214286) / 7451.742355998413 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.AppendAllText(size: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 5.455351259722223 < 6.355317062090772. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 71.74961311758135 (T) = (0 -5390.154767416302) / Math.Sqrt((31156.334830942546 / (299)) + (7421.354232356108 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.20368668817199653 = (6768.886928491442 - 5390.154767416302) / 6768.886928491442 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.AppendAllLinesAsync ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.698834392982457 < 15.311926653418801. IsChangePoint: Marked as a change because one of 2/23/2022 5:12:02 AM, 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 43.575019890644924 (T) = (0 -13223.116755383997) / Math.Sqrt((280191.580838147 / (299)) + (94868.6884210868 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.17612637553067495 = (16049.933342509046 - 13223.116755383997) / 16049.933342509046 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.WriteAllTextAsync(size: 10000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 24.02896778950217 < 28.288549711769893. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 74.84288350975262 (T) = (0 -24345.790381622683) / Math.Sqrt((605200.7635020916 / (299)) + (107136.61496139562 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1903725349583313 = (30070.361286928044 - 24345.790381622683) / 30070.361286928044 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.WriteAllText(size: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 66.3169841631356 < 69.27982712719667. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 38.02341453802494 (T) = (0 -66898.10054435929) / Math.Sqrt((1120683.9266166303 / (299)) + (821094.3276145868 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0923716463088128 = (73706.49040688828 - 66898.10054435929) / 73706.49040688828 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.CopyToOverwrite(size: 4096) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 42.92639400897791 < 45.48244087888163. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 26.578822983061134 (T) = (0 -42908.202098562106) / Math.Sqrt((1531174.1930929224 / (299)) + (810624.4392119909 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1012457487839987 = (47741.86274001813 - 42908.202098562106) / 47741.86274001813 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.CopyTo(size: 4096) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.71909512219353 < 26.170987259545583. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 64.49092478867051 (T) = (0 -21488.571486097517) / Math.Sqrt((932572.3883909695 / (299)) + (180324.6799407693 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.2268602616212178 = (27793.903765905874 - 21488.571486097517) / 27793.903765905874 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.WriteAllBytesAsync(size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 38.59978793616584 < 40.9509282537515. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 19.842185266500508 (T) = (0 -39176.64180861817) / Math.Sqrt((3359703.9606219865 / (299)) + (1325342.8139453148 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.10919059200728812 = (43978.702354408335 - 39176.64180861817) / 43978.702354408335 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.AppendAllLines ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 7.470408045673078 < 8.824726515442707. IsChangePoint: Marked as a change because one of 2/23/2022 5:12:02 AM, 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 83.45464243173194 (T) = (0 -7394.471630263594) / Math.Sqrt((70931.34831258676 / (299)) + (11144.39407793769 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.219582301676639 = (9475.017860499293 - 7394.471630263594) / 9475.017860499293 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.AppendAllTextAsync(size: 10000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 15.758625098901096 < 17.600122695337298. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 44.77921459541114 (T) = (0 -15119.382234946495) / Math.Sqrt((271383.70716822706 / (299)) + (124422.90420372077 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.17595945298800575 = (18347.861995104504 - 15119.382234946495) / 18347.861995104504 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.WriteAllBytesAsync(size: 4096) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 38.55028237336601 < 40.71687877008988. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 18.38726613012428 (T) = (0 -39446.14302880886) / Math.Sqrt((3787841.3867753167 / (299)) + (1675125.3300742488 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.11008858104293606 = (44325.92074730085 - 39446.14302880886) / 44325.92074730085 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.WriteAllText(size: 10000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.253677950048264 < 25.665587287654752. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 55.35481617065288 (T) = (0 -21387.342308937797) / Math.Sqrt((344228.0174905944 / (299)) + (240843.65631731274 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.20107694669226067 = (26770.215504971227 - 21387.342308937797) / 26770.215504971227 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.AppendAllTextAsync(size: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 7.877541687500002 < 9.474350053691962. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 31.510653386944124 (T) = (0 -7642.260483224912) / Math.Sqrt((230733.74217938053 / (299)) + (98544.45873195093 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.21026729288229545 = (9677.021623071618 - 7642.260483224912) / 9677.021623071618 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.AppendAllText(size: 10000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.213587299278847 < 13.085346681049105. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 76.70634574901264 (T) = (0 -11355.715101128768) / Math.Sqrt((161027.74959037406 / (299)) + (20795.356274310117 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.19313087663479736 = (14073.800536284716 - 11355.715101128768) / 14073.800536284716 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.IO.Tests.Perf_File.CopyTo(size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.430589557718584 < 26.1343549713903. IsChangePoint: Marked as a change because one of 4/27/2022 8:46:50 AM, 5/3/2022 11:42:39 AM falls between 4/24/2022 10:52:31 PM and 5/3/2022 11:42:39 AM. IsImprovementStdDev: Marked as improvement because 67.58550588570793 (T) = (0 -21523.837384426693) / Math.Sqrt((891861.5700813532 / (299)) + (158500.18352280333 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.22397327068747355 = (27735.948481432366 - 21523.837384426693) / 27735.948481432366 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
AndyAyersMS commented 2 years ago

Seems like clear improvement but the commit range doesn't include anything obvious.