search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 13 Regressions on 9/26/2022 7:45:50 PM #8791

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 315bdd4c138491df154e84b98ba7fae72ca8b83b
Compare 62db3613f21bca007e5b30af8342db32858ed2b9
Diff Diff

Regressions in System.IO.Tests.Perf_FileStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteByte - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.WriteByte(fileSize%3a%201024%2c%20options%3a%20Asynchronous).html>) 22.59 μs 31.23 μs 1.38 0.15 False
[Write - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.Write(fileSize%3a%201048576%2c%20userBufferSize%3a%20512%2c%20options%3a%20None).html>) 912.84 μs 1.01 ms 1.11 0.08 False
[Write_NoBuffering - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.Write_NoBuffering(fileSize%3a%201048576%2c%20userBufferSize%3a%2016384%2c%20options%3a%20Asynchronous).html>) 629.59 μs 730.23 μs 1.16 0.11 False
[Write - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.Write(fileSize%3a%201024%2c%20userBufferSize%3a%201024%2c%20options%3a%20None).html>) 7.92 μs 15.62 μs 1.97 0.26 False
[Flush - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.Flush(fileSize%3a%201024%2c%20options%3a%20None).html>) 477.32 μs 507.90 μs 1.06 0.09 False
[WriteByte - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.WriteByte(fileSize%3a%201024%2c%20options%3a%20None).html>) 21.48 μs 28.81 μs 1.34 0.21 False
[Flush - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.Flush(fileSize%3a%201024%2c%20options%3a%20Asynchronous).html>) 469.39 μs 516.23 μs 1.10 0.17 False
[Write_NoBuffering_PreallocationSize - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.Write_NoBuffering_PreallocationSize(fileSize%3a%201048576%2c%20userBufferSize%3a%2016384%2c%20options%3a%20None).html>) 647.33 μs 778.72 μs 1.20 0.07 False
[Write - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.Write(fileSize%3a%201024%2c%20userBufferSize%3a%201024%2c%20options%3a%20Asynchronous).html>) 7.88 μs 15.04 μs 1.91 0.22 False
[Write - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.Write(fileSize%3a%201048576%2c%20userBufferSize%3a%20512%2c%20options%3a%20Asynchronous).html>) 891.42 μs 1.01 ms 1.14 0.08 False
[Write_NoBuffering - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.Write_NoBuffering(fileSize%3a%201048576%2c%20userBufferSize%3a%2016384%2c%20options%3a%20None).html>) 659.22 μs 717.06 μs 1.09 0.07 False
[Write - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.Write(fileSize%3a%201048576%2c%20userBufferSize%3a%204096%2c%20options%3a%20Asynchronous).html>) 729.04 μs 798.82 μs 1.10 0.08 False
[CopyToFile - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileStream.CopyToFile(fileSize%3a%201024%2c%20options%3a%20None).html>) 21.45 μs 27.42 μs 1.28 0.19 False

graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_FileStream*'
### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.IO.Tests.Perf_FileStream.WriteByte(fileSize: 1024, options: Asynchronous) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 31.226490472661393 > 22.872463455061. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/26/2022 4:08:25 PM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -35.73432315998113 (T) = (0 -29714.324322627297) / Math.Sqrt((888104.2199976715 / (105)) + (892898.5700810059 / (23))) is less than -1.9789706019903577 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (105) + (23) - 2, .025) and -0.35408055262724936 = (21944.281132296153 - 29714.324322627297) / 21944.281132296153 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.Write(fileSize: 1048576, userBufferSize: 512, options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.01383671875 > 958.1962770220589. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/23/2022 4:51:48 AM, 9/27/2022 7:01:57 AM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -4.27633070238866 (T) = (0 -1012747.4285591735) / Math.Sqrt((12654528348.565815 / (107)) + (300354684.5140164 / (19))) is less than -1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (107) + (19) - 2, .025) and -0.051406036037196096 = (963231.514607117 - 1012747.4285591735) / 963231.514607117 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.Write_NoBuffering(fileSize: 1048576, userBufferSize: 16384, options: Asynchronous) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 730.2346580492423 > 673.5832030343751. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/23/2022 4:51:48 AM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -12.126635312023941 (T) = (0 -764755.7646998854) / Math.Sqrt((189523312.5063958 / (85)) + (3775966750.051372 / (40))) is less than -1.9794386850931271 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (85) + (40) - 2, .025) and -0.18465590360972153 = (645550.9674747124 - 764755.7646998854) / 645550.9674747124 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.Write(fileSize: 1024, userBufferSize: 1024, options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 15.621681541708043 > 8.452019671667413. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/26/2022 4:08:25 PM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -77.21463018331086 (T) = (0 -15161.628057091042) / Math.Sqrt((177803.96168991656 / (104)) + (155212.00460019565 / (23))) is less than -1.9791241094234997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (23) - 2, .025) and -0.8810019976239631 = (8060.399763659395 - 15161.628057091042) / 8060.399763659395 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.Flush(fileSize: 1024, options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 507.9044434027777 > 505.2187499311968. IsChangePoint: Marked as a change because one of 7/19/2022 2:55:40 PM, 9/26/2022 4:08:25 PM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -12.72419759802703 (T) = (0 -518339.5983514039) / Math.Sqrt((135782913.7919605 / (104)) + (94430447.71178238 / (22))) is less than -1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (104) + (22) - 2, .025) and -0.06166157403472385 = (488234.3027463199 - 518339.5983514039) / 488234.3027463199 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.WriteByte(fileSize: 1024, options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 28.809882528580026 > 22.802714223887463. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/26/2022 12:57:03 PM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -16.022377050447222 (T) = (0 -29733.913659216294) / Math.Sqrt((680900.3059752806 / (102)) + (5395479.046550263 / (23))) is less than -1.9794386850931271 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (23) - 2, .025) and -0.35994964132174806 = (21863.98139736823 - 29733.913659216294) / 21863.98139736823 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.Flush(fileSize: 1024, options: Asynchronous) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 516.2255949404762 > 506.4852794455788. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/23/2022 2:49:56 PM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -7.608340447126845 (T) = (0 -513918.3743774702) / Math.Sqrt((347837916.27608204 / (86)) + (247237687.60375088 / (39))) is less than -1.9794386850931271 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (86) + (39) - 2, .025) and -0.05009663375909673 = (489401.0302059196 - 513918.3743774702) / 489401.0302059196 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.Write_NoBuffering_PreallocationSize(fileSize: 1048576, userBufferSize: 16384, options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 778.7230798076923 > 704.335252993311. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/23/2022 4:51:48 AM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -16.266676899818616 (T) = (0 -739402.3855761553) / Math.Sqrt((228868549.11139008 / (87)) + (777637228.0003126 / (41))) is less than -1.9789706019903577 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (87) + (41) - 2, .025) and -0.1138829281107943 = (663806.1926581654 - 739402.3855761553) / 663806.1926581654 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.Write(fileSize: 1024, userBufferSize: 1024, options: Asynchronous) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 15.039386428796472 > 8.595414767257054. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/26/2022 4:08:25 PM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -55.680218639292654 (T) = (0 -15293.982732738636) / Math.Sqrt((364323.3548711089 / (106)) + (286093.7690084266 / (22))) is less than -1.9789706019903577 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (106) + (22) - 2, .025) and -0.8755340337336305 = (8154.468251526668 - 15293.982732738636) / 8154.468251526668 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.Write(fileSize: 1048576, userBufferSize: 512, options: Asynchronous) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.0142854347098216 > 942.3637062239585. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/23/2022 4:51:48 AM, 9/27/2022 7:01:57 AM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -4.688320427530535 (T) = (0 -1017447.9188445497) / Math.Sqrt((12652965514.76365 / (108)) + (279742430.8681873 / (19))) is less than -1.9791241094234997 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (108) + (19) - 2, .025) and -0.05587366254552624 = (963607.6312308626 - 1017447.9188445497) / 963607.6312308626 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.Write_NoBuffering(fileSize: 1048576, userBufferSize: 16384, options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 717.0610797483765 > 676.2721333229166. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/23/2022 4:51:48 AM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -11.785991502424027 (T) = (0 -736520.0063298538) / Math.Sqrt((122206640.19285889 / (86)) + (2424504124.099694 / (40))) is less than -1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (86) + (40) - 2, .025) and -0.14421207722801893 = (643691.8653351007 - 736520.0063298538) / 643691.8653351007 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.Write(fileSize: 1048576, userBufferSize: 4096, options: Asynchronous) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 798.8249995192307 > 754.2060290639049. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/23/2022 4:51:48 AM, 9/26/2022 12:57:03 PM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -4.065326830951006 (T) = (0 -823419.7460986457) / Math.Sqrt((9873742424.219923 / (100)) + (2976070062.682665 / (21))) is less than -1.9800998764566142 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (100) + (21) - 2, .025) and -0.08290515960919888 = (760380.2962724854 - 823419.7460986457) / 760380.2962724854 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.Perf_FileStream.CopyToFile(fileSize: 1024, options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 27.421586927536232 > 20.668644439307073. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/26/2022 4:08:25 PM, 9/30/2022 12:37:41 PM falls between 8/31/2022 10:21:55 AM and 9/30/2022 12:37:41 PM. IsRegressionStdDev: Marked as regression because -45.05783347090093 (T) = (0 -27764.255555556803) / Math.Sqrt((635193.7254263933 / (105)) + (544324.755405534 / (23))) is less than -1.9789706019903577 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (105) + (23) - 2, .025) and -0.38841621783309105 = (19997.069465875753 - 27764.255555556803) / 19997.069465875753 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 315bdd4c138491df154e84b98ba7fae72ca8b83b
Compare 62db3613f21bca007e5b30af8342db32858ed2b9
Diff Diff

Regressions in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteAllBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_File.WriteAllBytes(size%3a%20512).html>) 6.36 μs 6.95 μs 1.09 0.26 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'
### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.IO.Tests.Perf_File.WriteAllBytes(size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.950586489786856 > 6.868885872574335. IsChangePoint: Marked as a change because one of 7/19/2022 10:33:10 AM, 9/26/2022 4:08:25 PM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsRegressionStdDev: Marked as regression because -32.689147929880946 (T) = (0 -12569.508736355301) / Math.Sqrt((277888.07896964625 / (24)) + (974351.2138440278 / (38))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (38) - 2, .025) and -1.006946090842092 = (6263.002675413806 - 12569.508736355301) / 6263.002675413806 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)