search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 16 Improvements on 9/12/2023 2:43:35 PM #21786

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 9d08b24d743d0c57203a55c3d0c6dd1bc472e57e
Compare 2fe26a11b1185b3793c7c653f6b07d91c668ff80
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.27 ns 4.75 ns 0.76 0.29 False
113.30 ns 83.24 ns 0.73 0.08 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_String.Trim_CharArr(s: "Test", c: [' ', ' ']) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.753913258763015 < 5.7107453426132295. IsChangePoint: Marked as a change because one of 8/27/2023 2:42:29 PM, 9/2/2023 3:43:46 AM, 9/12/2023 12:17:30 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 7.34221584467189 (T) = (0 -4.881906921479839) / Math.Sqrt((0.3180622817277046 / (103)) + (0.09427885782732551 / (24))) is greater than 1.9791241094234981 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (103) + (24) - 2, .975) and 0.11188227942926601 = (5.496914213515032 - 4.881906921479839) / 5.496914213515032 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.IndexerCheckPathLength #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 83.23609564369629 < 107.50657826413614. IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 8.533682454919767 (T) = (0 -83.07274738402093) / Math.Sqrt((208.1881883847244 / (102)) + (0.05370898468774405 / (24))) is greater than 1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (24) - 2, .975) and 0.12803864131238935 = (95.2711339285192 - 83.07274738402093) / 95.2711339285192 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline da4e544809b3b10b7db8dc170331988d817240d7
Compare 2fe26a11b1185b3793c7c653f6b07d91c668ff80
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Perf_Frozen<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
14.46 ns 10.17 ns 0.70 0.38 False
1.84 μs 1.54 μs 0.84 0.20 True
209.83 ns 174.07 ns 0.83 0.42 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.165734729950277 < 13.723421528440408. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 15.357258083450763 (T) = (0 -10.153852051245494) / Math.Sqrt((1.3178473292467388 / (35)) + (0.036091781720488346 / (24))) is greater than 2.002465459290109 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (24) - 2, .975) and 0.23034609097002973 = (13.19275057544092 - 10.153852051245494) / 13.19275057544092 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.5432793484140344 < 1.6986048222591419. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 23.020804539920253 (T) = (0 -1553.0314159370002) / Math.Sqrt((4238.976990533607 / (35)) + (24.15138185549449 / (24))) is greater than 2.002465459290109 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (24) - 2, .975) and 0.14075130255410623 = (1807.4294678052665 - 1553.0314159370002) / 1807.4294678052665 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<Int16>.TryGetValue_True(Count: 64) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 174.07194281479693 < 206.13747891700098. IsChangePoint: Marked as a change because one of 9/15/2023 8:45:09 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 12.103187132898013 (T) = (0 -177.1124473181532) / Math.Sqrt((73.94053830664433 / (50)) + (88.85023255479356 / (10))) is greater than 2.0017174841444287 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (50) + (10) - 2, .975) and 0.18032385546261162 = (216.0761277469073 - 177.1124473181532) / 216.0761277469073 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline da4e544809b3b10b7db8dc170331988d817240d7
Compare 2fe26a11b1185b3793c7c653f6b07d91c668ff80
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Perf_SubstringFrozenDictionary

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.51 μs 6.19 μs 0.82 0.43 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_SubstringFrozenDictionary*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Perf_SubstringFrozenDictionary.TryGetValue_True_FrozenDictionary(Count: 1000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.1853389950248046 < 7.174405158583463. IsChangePoint: Marked as a change because one of 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 12.997725052616069 (T) = (0 -6023.792853249281) / Math.Sqrt((283125.39910982735 / (35)) + (50863.17329630168 / (24))) is greater than 2.002465459290109 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (24) - 2, .975) and 0.17899000153846883 = (7337.051758854467 - 6023.792853249281) / 7337.051758854467 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline da4e544809b3b10b7db8dc170331988d817240d7
Compare 2fe26a11b1185b3793c7c653f6b07d91c668ff80
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.XposMatrix

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.16 μs 15.14 μs 0.88 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.XposMatrix*'
### Payloads [Baseline]() [Compare]() ### Benchstone.BenchI.XposMatrix.Test #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.141084627950702 < 16.281627385025576. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 17.30131135755746 (T) = (0 -15124.0522628197) / Math.Sqrt((783546.0564192389 / (102)) + (253.24850206954477 / (24))) is greater than 1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (24) - 2, .975) and 0.091183700827713 = (16641.484397445416 - 15124.0522628197) / 16641.484397445416 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline da4e544809b3b10b7db8dc170331988d817240d7
Compare 2fe26a11b1185b3793c7c653f6b07d91c668ff80
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.22 μs 1.28 μs 0.57 0.16 False
894.12 ns 644.88 ns 0.72 0.10 False
161.14 ns 122.58 ns 0.76 0.18 False
420.48 ns 244.03 ns 0.58 0.08 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'
### Payloads [Baseline]() [Compare]() ### System.IO.Tests.Perf_Path.GetFullPathForReallyLongPath #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2767339271830345 < 2.1062183544102187. IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 7.6278260050100535 (T) = (0 -1229.4970821752443) / Math.Sqrt((145305.75820758563 / (102)) + (123.82593165597156 / (24))) is greater than 1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (24) - 2, .975) and 0.1900110037213157 = (1517.9182529934324 - 1229.4970821752443) / 1517.9182529934324 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.Perf_Path.GetFullPathForTypicalLongPath #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 644.8777965865552 < 866.5880859312206. IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 7.834687407329408 (T) = (0 -644.4116879031832) / Math.Sqrt((19556.86377767385 / (102)) + (13.503132539316233 / (25))) is greater than 1.9791241094234981 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (25) - 2, .975) and 0.14426413449264308 = (753.049759718927 - 644.4116879031832) / 753.049759718927 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.Perf_Path.GetFullPathNoRedundantSegments #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 122.57842661991783 < 153.1468813331593. IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 12.153390119962989 (T) = (0 -121.9464209782906) / Math.Sqrt((436.4793325480247 / (101)) + (1.0365778887874246 / (25))) is greater than 1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (101) + (25) - 2, .975) and 0.17230350984187745 = (147.33229200355078 - 121.9464209782906) / 147.33229200355078 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.Perf_Path.GetFullPathForLegacyLength #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 244.03208000538805 < 340.44518715799455. IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 8:56:48 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 8.743449056108803 (T) = (0 -243.57635827861458) / Math.Sqrt((3014.9815781629873 / (103)) + (1.0055192439024088 / (25))) is greater than 1.978970601990356 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (103) + (25) - 2, .975) and 0.16271978620308772 = (290.9137876004981 - 243.57635827861458) / 290.9137876004981 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline da4e544809b3b10b7db8dc170331988d817240d7
Compare 2fe26a11b1185b3793c7c653f6b07d91c668ff80
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.47 μs 5.91 μs 0.91 0.18 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CreateAddAndClear<Int32>.ConcurrentStack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.910495264268224 < 6.135047013332808. IsChangePoint: Marked as a change because one of 9/4/2023 7:24:21 PM, 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 11.483943766836909 (T) = (0 -5881.8721493844905) / Math.Sqrt((74367.42315853554 / (102)) + (5231.3909051761875 / (24))) is greater than 1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (24) - 2, .975) and 0.05667938824126111 = (6235.284245955629 - 5881.8721493844905) / 6235.284245955629 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline da4e544809b3b10b7db8dc170331988d817240d7
Compare 2fe26a11b1185b3793c7c653f6b07d91c668ff80
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
131.10 μs 109.38 μs 0.83 0.03 False
180.01 μs 152.89 μs 0.85 0.35 False
131.46 μs 109.55 μs 0.83 0.05 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.LowLevelPerf.SealedClassInterfaceMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 109.37821672453705 < 124.73173541703702. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 17.636629085217063 (T) = (0 -109465.32474116689) / Math.Sqrt((89556073.65888356 / (102)) + (37662.171557669564 / (24))) is greater than 1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (24) - 2, .975) and 0.1312683049567814 = (126005.90650226142 - 109465.32474116689) / 126005.90650226142 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.GenericClassWithSTringGenericInstanceMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 152.8890969786953 < 171.60124813710146. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 5.485093658796381 (T) = (0 -155111.4516573457) / Math.Sqrt((150824869.93927813 / (102)) + (54599531.82029776 / (24))) is greater than 1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (24) - 2, .975) and 0.0641191785202968 = (165738.46594281305 - 155111.4516573457) / 165738.46594281305 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 109.55011925574422 < 124.87996053728263. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 8/28/2023 1:23:07 PM, 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 14.58878369227376 (T) = (0 -110157.81848174834) / Math.Sqrt((90375426.27830741 / (102)) + (7292418.636962785 / (24))) is greater than 1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (24) - 2, .975) and 0.12622763435192497 = (126071.52939661182 - 110157.81848174834) / 126071.52939661182 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline da4e544809b3b10b7db8dc170331988d817240d7
Compare 2fe26a11b1185b3793c7c653f6b07d91c668ff80
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.Perf_FileInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
106.69 ns 76.63 ns 0.72 0.22 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileInfo*'
### Payloads [Baseline]() [Compare]() ### System.IO.Tests.Perf_FileInfo.ctor_str #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 76.632613237488 < 101.34663179967302. IsChangePoint: Marked as a change because one of 9/2/2023 3:43:46 AM, 9/11/2023 3:39:38 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsImprovementStdDev: Marked as improvement because 11.285513905180887 (T) = (0 -74.75091867392457) / Math.Sqrt((102.80042556848186 / (102)) + (1.6758580112007262 / (24))) is greater than 1.9792801166046092 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (102) + (24) - 2, .975) and 0.13549308366039056 = (86.466536312313 - 74.75091867392457) / 86.466536312313 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)