search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 6/9/2022 10:25:37 PM #6027

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3d74b00659fec817506e2888f87936518556e01c
Compare b2add4986c2da96e6e5fbf6f1fbb947cc7e33b42
Diff Diff

Regressions in System.Threading.Tasks.Tests.Perf_AsyncMethods

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SingleYieldMethodInvocation - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Threading.Tasks.Tests.Perf_AsyncMethods.SingleYieldMethodInvocation.html>) 494.25 ns 535.52 ns 1.08 0.08 False
[Yield - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Threading.Tasks.Tests.Perf_AsyncMethods.Yield.html>) 321.86 ns 379.62 ns 1.18 0.06 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Tasks.Tests.Perf_AsyncMethods*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Tasks.Tests.Perf_AsyncMethods.SingleYieldMethodInvocation ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 535.5206152777778 > 520.9610272505981. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/9/2022 8:09:33 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -22.48018346374809 (T) = (0 -533.5893804455385) / Math.Sqrt((75.6693884576559 / (48)) + (74.10705241539148 / (27))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (27) - 2, .025) and -0.09598346138853392 = (486.85897118330445 - 533.5893804455385) / 486.85897118330445 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Threading.Tasks.Tests.Perf_AsyncMethods.Yield ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 379.62419400000005 > 341.4747679. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/9/2022 8:09:33 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -17.76400536321322 (T) = (0 -354.6966585346222) / Math.Sqrt((34.881956520528384 / (48)) + (57.09328807906058 / (27))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (27) - 2, .025) and -0.09220268550706515 = (324.75351254968854 - 354.6966585346222) / 324.75351254968854 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3d74b00659fec817506e2888f87936518556e01c
Compare b2add4986c2da96e6e5fbf6f1fbb947cc7e33b42
Diff Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SingleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Tests.RentReturnArrayPoolTests(Byte).SingleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20False%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20False).html>) 368.52 ns 403.20 ns 1.09 0.04 False
[SingleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Tests.RentReturnArrayPoolTests(Byte).SingleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20False%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20True).html>) 361.10 ns 421.39 ns 1.17 0.10 False
[ProducerConsumer - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Tests.RentReturnArrayPoolTests(Byte).ProducerConsumer(RentalSize%3a%204096%2c%20ManipulateArray%3a%20False%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20False).html>) 608.70 ns 650.28 ns 1.07 0.03 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 403.2018894444444 > 387.65877389166667. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/9/2022 8:09:33 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -22.945322343920836 (T) = (0 -408.9138240677995) / Math.Sqrt((23.876015691936963 / (48)) + (50.621799470007424 / (27))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (27) - 2, .025) and -0.09460270402190157 = (373.57282470189995 - 408.9138240677995) / 373.57282470189995 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 421.3889634444444 > 382.40218269843746. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/9/2022 8:09:33 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -20.73757823468293 (T) = (0 -413.5076548591558) / Math.Sqrt((32.90079304928567 / (48)) + (121.9751562820474 / (27))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (27) - 2, .025) and -0.129170129619781 = (366.2049181184007 - 413.5076548591558) / 366.2049181184007 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.ProducerConsumer(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 650.2766078846154 > 649.48797025. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/9/2022 8:09:33 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -14.823913212125632 (T) = (0 -648.8817466225918) / Math.Sqrt((87.5550169471844 / (48)) + (106.33057315718916 / (27))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (27) - 2, .025) and -0.05802124921117852 = (613.2974617536028 - 648.8817466225918) / 613.2974617536028 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3d74b00659fec817506e2888f87936518556e01c
Compare b2add4986c2da96e6e5fbf6f1fbb947cc7e33b42
Diff Diff

Regressions in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.ContainsTrueComparer(Int32).HashSet(Size%3a%20512).html>) 4.37 μs 5.18 μs 1.18 0.22 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrueComparer<Int32>.HashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.182157971771166 > 4.597610556012575. IsChangePoint: Marked as a change because one of 4/15/2022 9:40:04 PM, 4/29/2022 5:18:09 PM, 5/23/2022 7:50:08 PM, 5/24/2022 7:55:26 PM, 6/9/2022 8:09:33 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -11.861112658412102 (T) = (0 -5006.401593127916) / Math.Sqrt((10615.446673983432 / (48)) + (78315.59987425832 / (27))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (27) - 2, .025) and -0.1525682927256445 = (4343.6919310772955 - 5006.401593127916) / 4343.6919310772955 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3d74b00659fec817506e2888f87936518556e01c
Compare b2add4986c2da96e6e5fbf6f1fbb947cc7e33b42
Diff Diff

Regressions in System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[PingPong - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.PingPong.html>) 11.15 ms 12.31 ms 1.10 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Channels.Tests.SpscUnboundedChannelPerfTests.PingPong ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.314238643598618 > 11.897402113782055. IsChangePoint: Marked as a change because one of 4/18/2022 10:15:32 PM, 6/9/2022 8:09:33 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -12.370843587539024 (T) = (0 -12171274.297561642) / Math.Sqrt((46547730773.25141 / (48)) + (79147121750.46172 / (27))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (27) - 2, .025) and -0.06778634919619166 = (11398604.511777038 - 12171274.297561642) / 11398604.511777038 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3d74b00659fec817506e2888f87936518556e01c
Compare b2add4986c2da96e6e5fbf6f1fbb947cc7e33b42
Diff Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SingleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Tests.RentReturnArrayPoolTests(Object).SingleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20False%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20True).html>) 377.86 ns 409.84 ns 1.08 0.04 False
[ProducerConsumer - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Tests.RentReturnArrayPoolTests(Object).ProducerConsumer(RentalSize%3a%204096%2c%20ManipulateArray%3a%20False%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20True).html>) 424.22 ns 454.09 ns 1.07 0.38 False
[SingleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Tests.RentReturnArrayPoolTests(Object).SingleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20False%2c%20Async%3a%20True%2c%20UseSharedPool%3a%20False).html>) 380.98 ns 419.81 ns 1.10 0.02 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 409.8398430952381 > 393.8729809401042. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/9/2022 8:09:33 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -25.92296322597156 (T) = (0 -412.6578066201368) / Math.Sqrt((46.718836450054425 / (48)) + (21.044486731070485 / (27))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (27) - 2, .025) and -0.09071154396391452 = (378.3381673218903 - 412.6578066201368) / 378.3381673218903 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.ProducerConsumer(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 454.0891785 > 449.9570260799343. IsChangePoint: Marked as a change because one of 6/9/2022 8:09:33 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -7.418329456045893 (T) = (0 -462.8399061199834) / Math.Sqrt((422.9275053585104 / (48)) + (293.3920555518497 / (27))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (27) - 2, .025) and -0.07654008961794229 = (429.9328102906438 - 462.8399061199834) / 429.9328102906438 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 419.8114596666667 > 399.7841063708334. IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 6/9/2022 8:09:33 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -32.041276562000164 (T) = (0 -415.7748347855108) / Math.Sqrt((22.901519137454354 / (48)) + (17.217923078916435 / (27))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (27) - 2, .025) and -0.08857508233600116 = (381.94410429943787 - 415.7748347855108) / 381.94410429943787 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
mrsharm commented 2 years ago

Referenced Regression details here for the August Perf Report: https://github.com/dotnet/perf-autofiling-issues/issues/6009#issuecomment-1209979413