dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/arm64: 3 Improvements on 6/2/2023 3:25:13 AM #18600

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 19b1c9a7a122606bfa6e30a64d8aa8192391c79a
Compare 2855dd2d29c64df89498fbcb984653f596c1b976
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MultipleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Object).MultipleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20False%2c%20Async%3a%20False%2c%20UseSharedPool%3a%20False).html>) 348.39 ns 315.53 ns 0.91 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/1/2023 5:52:46 AM, 6/7/2023 7:43:49 PM falls between 5/29/2023 11:58:32 PM and 6/7/2023 7:43:49 PM. IsImprovementStdDev: Marked as improvement because 9.523909252062573 (T) = (0 -323.69209593739595) / Math.Sqrt((120.39152402357809 / (299)) + (313.26976773505 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.13653375182624683 = (374.87521558834595 - 323.69209593739595) / 374.87521558834595 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 19b1c9a7a122606bfa6e30a64d8aa8192391c79a
Compare 2855dd2d29c64df89498fbcb984653f596c1b976
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tasks.ValueTaskPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateAndAwait_FromCompletedValueTaskSource_ConfigureAwait - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedValueTaskSource_ConfigureAwait.html>) 24.30 ns 19.43 ns 0.80 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Tasks.ValueTaskPerfTest.CreateAndAwait_FromCompletedValueTaskSource_ConfigureAwait ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 6/1/2023 5:52:46 AM, 6/7/2023 7:43:49 PM falls between 5/29/2023 11:58:32 PM and 6/7/2023 7:43:49 PM. IsImprovementStdDev: Marked as improvement because 58.841937178304 (T) = (0 -19.536909385100504) / Math.Sqrt((3.3386042741391253 / (299)) + (0.14388122162034805 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.3192540026456117 = (28.699264426126064 - 19.536909385100504) / 28.699264426126064 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 19b1c9a7a122606bfa6e30a64d8aa8192391c79a
Compare 2855dd2d29c64df89498fbcb984653f596c1b976
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MultipleSerial - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_Windows 10.0.19041/System.Buffers.Tests.RentReturnArrayPoolTests(Byte).MultipleSerial(RentalSize%3a%204096%2c%20ManipulateArray%3a%20False%2c%20Async%3a%20False%2c%20UseSharedPool%3a%20False).html>) 345.12 ns 315.96 ns 0.92 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 6/1/2023 5:52:46 AM, 6/7/2023 7:43:49 PM falls between 5/29/2023 11:58:32 PM and 6/7/2023 7:43:49 PM. IsImprovementStdDev: Marked as improvement because 90.29182018800874 (T) = (0 -314.4305104895105) / Math.Sqrt((55.128506887767564 / (299)) + (0.8332070856351326 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.12774811484063742 = (360.48131948956836 - 314.4305104895105) / 360.48131948956836 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
EgorBo commented 1 year ago

https://github.com/dotnet/runtime/pull/86225