search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 143 Improvements on 8/24/2023 4:35:49 AM #21099

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
20.90 μs 19.13 μs 0.92 0.06 True
56.42 μs 53.27 μs 0.94 0.02 True
20.86 μs 19.56 μs 0.94 0.02 True
26.84 μs 24.63 μs 0.92 0.08 True
53.91 μs 49.80 μs 0.92 0.04 False
26.34 μs 20.84 μs 0.79 0.02 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'
### Payloads [Baseline]() [Compare]() ### System.MathBenchmarks.Single.ScaleB #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.130640531994054 < 19.869544594784756. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 18.184889224967723 (T) = (0 -19251.801403988433) / Math.Sqrt((9852.563695592982 / (27)) + (186817.42762639935 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.08163707140858811 = (20963.173495599294 - 19251.801403988433) / 20963.173495599294 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.MathBenchmarks.Single.TanPi #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 53.27288332118562 < 53.45131862587567. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 31.492790679615982 (T) = (0 -53317.1341256909) / Math.Sqrt((249890.11279903862 / (27)) + (26601.324790540446 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.0569798439838132 = (56538.70045676491 - 53317.1341256909) / 56538.70045676491 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.MathBenchmarks.Single.ILogB #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 19.556617132442287 < 19.95232563693267. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 18.974387290074812 (T) = (0 -19333.01061652158) / Math.Sqrt((53030.194451720534 / (27)) + (144059.36051373862 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.08174589150907956 = (21054.096505262456 - 19333.01061652158) / 21054.096505262456 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.MathBenchmarks.Single.Hypot #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 24.628874962084442 < 25.52607473291367. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.594457714606689 (T) = (0 -24544.01057944347) / Math.Sqrt((1872531.6739196854 / (26)) + (3771.834491033662 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.09504389366416971 = (27121.76911963414 - 24544.01057944347) / 27121.76911963414 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.MathBenchmarks.Single.Cbrt #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 49.79932041913102 < 51.222931914655646. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.556853025878706 (T) = (0 -49734.80790200459) / Math.Sqrt((4712712.055855442 / (27)) + (10634.482319371233 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.06714242208296325 = (53314.47058945126 - 49734.80790200459) / 53314.47058945126 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.MathBenchmarks.Single.Round #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.83719252601703 < 25.011271234284674. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 149.72930738807918 (T) = (0 -20877.316100832893) / Math.Sqrt((28277.66712048076 / (26)) + (5757.688939211705 / (20))) is greater than 2.015367574442192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (20) - 2, .975) and 0.21010246850109285 = (26430.410614420027 - 20877.316100832893) / 26430.410614420027 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare fd6d5fd0a330a8a6fd286a3d924f4e832aaeb479
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.CreateAddAndRemove<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
34.45 μs 30.56 μs 0.89 0.42 False
78.60 μs 72.43 μs 0.92 0.12 False
22.30 μs 17.07 μs 0.77 0.45 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CreateAddAndRemove<String>.LinkedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 30.560904422949733 < 32.385093717115275. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.216458328243135 (T) = (0 -31668.303022844506) / Math.Sqrt((4250378.364412071 / (26)) + (1019116.2220134105 / (22))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (22) - 2, .975) and 0.08249523171358648 = (34515.68222581569 - 31668.303022844506) / 34515.68222581569 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndRemove<String>.HashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 72.42554006558642 < 73.87407247030843. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.27342135029556 (T) = (0 -74043.0620427387) / Math.Sqrt((7674748.619871234 / (27)) + (1648591.1905936971 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.06249635768017152 = (78978.95933451636 - 74043.0620427387) / 78978.95933451636 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndRemove<String>.Stack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.06701006868132 < 18.010996545025815. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.010808020651864 (T) = (0 -17708.718930474424) / Math.Sqrt((1463798.5115963307 / (27)) + (1006504.3971094191 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.09691645873619079 = (19609.170271990755 - 17708.718930474424) / 19609.170271990755 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.Sort<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
836.48 μs 782.85 μs 0.94 0.22 True
865.70 μs 730.76 μs 0.84 0.13 True
860.44 μs 758.87 μs 0.88 0.36 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Sort<String>.Array_Comparison(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 782.8541668166667 < 791.7163940848973. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 13.784988710857249 (T) = (0 -757726.9811009078) / Math.Sqrt((361037716.1160496 / (25)) + (227539160.5519313 / (23))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .975) and 0.08235321587472212 = (825728.3676128072 - 757726.9811009078) / 825728.3676128072 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<String>.Array_ComparerClass(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 730.7633001714285 < 796.6460679358929. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 15.64465382538495 (T) = (0 -751155.3495979757) / Math.Sqrt((417568919.15320873 / (27)) + (144015830.25054395 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.08849075239423945 = (824078.6931904613 - 751155.3495979757) / 824078.6931904613 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<String>.Array_ComparerStruct(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 758.8726461538462 < 795.4421820379998. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 11.111393320967602 (T) = (0 -759266.5383397621) / Math.Sqrt((811534424.1948917 / (26)) + (508631456.7187791 / (22))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (22) - 2, .975) and 0.097367692015316 = (841169.2464620328 - 759266.5383397621) / 841169.2464620328 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.16 μs 3.74 μs 0.90 0.50 False
4.48 μs 3.77 μs 0.84 0.32 False
2.70 μs 2.49 μs 0.92 0.26 True
78.24 μs 70.01 μs 0.89 0.39 False
3.92 μs 3.14 μs 0.80 0.33 False
5.83 μs 5.07 μs 0.87 0.25 False
16.79 μs 15.66 μs 0.93 0.24 False
3.58 μs 3.24 μs 0.91 0.46 False
2.77 μs 2.45 μs 0.88 0.41 True
3.65 μs 3.32 μs 0.91 0.31 True
3.71 μs 3.24 μs 0.87 0.39 False
4.29 μs 4.03 μs 0.94 0.31 False
81.83 ns 71.34 ns 0.87 0.49 False
3.91 μs 3.43 μs 0.88 0.33 False
3.65 μs 3.37 μs 0.92 0.50 False
3.89 μs 3.24 μs 0.83 0.41 False
3.57 μs 3.25 μs 0.91 0.45 False
7.36 μs 6.13 μs 0.83 0.35 False
4.17 μs 3.44 μs 0.83 0.41 False
3.25 μs 3.02 μs 0.93 0.27 False
4.13 μs 3.71 μs 0.90 0.45 False
3.63 μs 3.31 μs 0.91 0.40 False
840.34 ns 774.07 ns 0.92 0.06 True
1.15 μs 1.06 μs 0.93 0.45 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.737286661971256 < 3.975220994968077. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.48742882271103 (T) = (0 -3878.7056251528365) / Math.Sqrt((46364.39385356471 / (27)) + (18945.377988833916 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.10976439110795054 = (4356.942798525116 - 3878.7056251528365) / 4356.942798525116 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.7717891885126114 < 4.17420895896795. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 12.208945216465015 (T) = (0 -3820.652361562388) / Math.Sqrt((38160.75242728188 / (25)) + (14789.285560158145 / (23))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .975) and 0.12955548067918782 = (4389.311755956089 - 3820.652361562388) / 4389.311755956089 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.488241107425184 < 2.650673306342793. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 12.380968197317321 (T) = (0 -2493.2071122142543) / Math.Sqrt((12827.349809069377 / (25)) + (6637.966718563053 / (23))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .975) and 0.12327273748528593 = (2843.765922212794 - 2493.2071122142543) / 2843.765922212794 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 70.00761066315972 < 74.43604824801089. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 5.238886890243211 (T) = (0 -73734.12082977839) / Math.Sqrt((7741421.088655233 / (27)) + (20161270.129837554 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.07229968781250984 = (79480.53898560785 - 73734.12082977839) / 79480.53898560785 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.All_AllElementsMatch(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.1380168705940803 < 3.501844623229077. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 11.272361883489493 (T) = (0 -3262.1173613756773) / Math.Sqrt((36043.26487238314 / (27)) + (11030.615999218579 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.1283096666064543 = (3742.289247003632 - 3262.1173613756773) / 3742.289247003632 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: IEnumerable, input2: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.073769911767889 < 5.28113028186986. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.427640910303014 (T) = (0 -5151.754080738229) / Math.Sqrt((86990.68722328728 / (27)) + (16650.751899717274 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.11280280650965935 = (5806.774546333502 - 5151.754080738229) / 5806.774546333502 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Where(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.66184272946339 < 15.881605964292262. IsChangePoint: Marked as a change because one of 8/10/2023 8:47:51 AM, 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 5.724420928538757 (T) = (0 -14809.792396812883) / Math.Sqrt((781376.474282978 / (27)) + (223031.0663129896 / (21))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (21) - 2, .975) and 0.07139135576357086 = (15948.368011360257 - 14809.792396812883) / 15948.368011360257 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.238541902698513 < 3.572379200787581. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.555603207892162 (T) = (0 -3335.1516679102588) / Math.Sqrt((49187.239040961686 / (27)) + (35235.01264948892 / (21))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (21) - 2, .975) and 0.13176023952757043 = (3841.279586292529 - 3335.1516679102588) / 3841.279586292529 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.4482722442286393 < 2.6404625828307715. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 12.16961096172164 (T) = (0 -2537.2893410230786) / Math.Sqrt((13113.689068477326 / (25)) + (8784.782676890283 / (24))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (24) - 2, .975) and 0.12521201908688948 = (2900.4620506727083 - 2537.2893410230786) / 2900.4620506727083 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Aggregate(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.317667150893684 < 3.506673960121012. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 13.536188077634574 (T) = (0 -3264.356795804637) / Math.Sqrt((20788.390779841786 / (27)) + (15492.861634035822 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.13610513908032962 = (3778.650555149181 - 3264.356795804637) / 3778.650555149181 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.2406853186172424 < 3.498149493086821. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 11.680847844693266 (T) = (0 -3229.291264703176) / Math.Sqrt((41037.24914531013 / (26)) + (20759.163733525067 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.15266199146206802 = (3811.101629059772 - 3229.291264703176) / 3811.101629059772 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.ToList(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.0286583736910995 < 4.085137270357189. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.970468137200571 (T) = (0 -3963.833302040247) / Math.Sqrt((53594.90270188795 / (27)) + (29457.589838701162 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.10306721846451769 = (4419.3203589397845 - 3963.833302040247) / 4419.3203589397845 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IList) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 71.33535385598901 < 76.40500189838275. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.0129602448121116 (T) = (0 -71.78082611057745) / Math.Sqrt((42.79171529957132 / (26)) + (9.670922179948107 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.10747291573784876 = (80.42425532656904 - 71.78082611057745) / 80.42425532656904 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.43230399127563 < 3.5566270905905095. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.819154005882647 (T) = (0 -3226.9803277057945) / Math.Sqrt((31204.8845207803 / (26)) + (16188.352573222499 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.1276253211699358 = (3699.0761034392654 - 3226.9803277057945) / 3699.0761034392654 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.372000163363103 < 3.5037666352494834. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.0512623284707585 (T) = (0 -3406.3213159463926) / Math.Sqrt((42408.10895826676 / (27)) + (51969.17364247052 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.11356369624850492 = (3842.7141369667165 - 3406.3213159463926) / 3842.7141369667165 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.2376185064935057 < 3.395264459939505. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.170606114543855 (T) = (0 -3282.460113607751) / Math.Sqrt((27450.91045557498 / (26)) + (22966.520958588015 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.11239693086885949 = (3698.117128888361 - 3282.460113607751) / 3698.117128888361 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Aggregate_Seed(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.2450644581022328 < 3.436324787009808. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.329703942155316 (T) = (0 -3292.6319182782654) / Math.Sqrt((45527.15893754838 / (28)) + (15898.58697660963 / (23))) is greater than 2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (23) - 2, .975) and 0.13120357851481854 = (3789.8773945795147 - 3292.6319182782654) / 3789.8773945795147 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.ToDictionary(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.132962737767284 < 6.5201259555434765. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.721333103132735 (T) = (0 -6216.662420047377) / Math.Sqrt((115419.38252788322 / (27)) + (65580.78455275833 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.09489846505898548 = (6868.469646836382 - 6216.662420047377) / 6868.469646836382 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.4440984063719995 < 3.5921892422860195. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 11.038802266035606 (T) = (0 -3233.271429779666) / Math.Sqrt((63771.816324856176 / (27)) + (13634.133557935696 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.15713298436753273 = (3836.039813888667 - 3233.271429779666) / 3836.039813888667 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.024339682539683 < 3.048413763974706. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.25119958942036 (T) = (0 -2953.0810888269702) / Math.Sqrt((24437.02014135697 / (25)) + (8452.881909544552 / (24))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (24) - 2, .975) and 0.11235970333842091 = (3326.889394198897 - 2953.0810888269702) / 3326.889394198897 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: List) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.7059484265182023 < 4.112647507826907. IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.602515062598776 (T) = (0 -3820.7494746184802) / Math.Sqrt((90944.0937170042 / (27)) + (10468.966324702385 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.12249721539914636 = (4354.116638337974 - 3820.7494746184802) / 4354.116638337974 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: Array) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.311751752989457 < 3.379499131503324. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 11.316086095611361 (T) = (0 -3210.2580463772438) / Math.Sqrt((35787.89835040685 / (27)) + (14562.420606819313 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.13495011496418968 = (3711.0669591550195 - 3210.2580463772438) / 3711.0669591550195 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 774.067735187895 < 782.1190232372368. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 14.43895399144575 (T) = (0 -756.0445627176259) / Math.Sqrt((325.60848721433246 / (25)) + (165.79527866217293 / (23))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .975) and 0.07910883004196032 = (820.992303305585 - 756.0445627176259) / 820.992303305585 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Linq.Tests.Perf_Enumerable.Count(input: IEnumerable) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0596883603076217 < 1.1148393746202283. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 4.715871040035632 (T) = (0 -1092.4282386274822) / Math.Sqrt((4480.327935552956 / (26)) + (5465.243845819257 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.08037818673585156 = (1187.9103158177236 - 1092.4282386274822) / 1187.9103158177236 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.TryAddGiventSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
90.99 μs 85.46 μs 0.94 0.26 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.TryAddGiventSize<Int32>.ConcurrentDictionary(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 85.46041708333333 < 87.14831314662966. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.335369845553809 (T) = (0 -86199.78860862646) / Math.Sqrt((14974751.087195752 / (25)) + (7475924.402712977 / (23))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .975) and 0.07561561784522922 = (93251.02227245757 - 86199.78860862646) / 93251.02227245757 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in GuardedDevirtualization.TwoClassVirtual

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.00 ns 4.98 ns 0.83 0.14 True
10.06 ns 8.70 ns 0.87 0.13 True
9.80 ns 8.25 ns 0.84 0.12 True
8.88 ns 7.54 ns 0.85 0.23 True
10.41 ns 8.79 ns 0.84 0.09 True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.TwoClassVirtual*'
### Payloads [Baseline]() [Compare]() ### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.980598333333333 < 5.5659053623344406. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 20.424273786327717 (T) = (0 -5.00372806510552) / Math.Sqrt((0.019314090767469882 / (26)) + (0.029960008885227485 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.15583821724061497 = (5.92745154696461 - 5.00372806510552) / 5.92745154696461 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.40) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.704856080790718 < 9.558579989205162. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 24.3371059796928 (T) = (0 -8.623670134889442) / Math.Sqrt((0.10028830958749758 / (27)) + (0.01573586253875744 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.15765922175052924 = (10.237745052318273 - 8.623670134889442) / 10.237745052318273 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.60) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.247375342860925 < 9.319668660309596. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 26.28514060439743 (T) = (0 -8.292735292294369) / Math.Sqrt((0.04633997825324039 / (26)) + (0.04324051337052641 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.16094628442305445 = (9.883437899553503 - 8.292735292294369) / 9.883437899553503 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.70) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.540426854478512 < 8.36067864652386. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 21.921423652878104 (T) = (0 -7.49710106802205) / Math.Sqrt((0.08115976530162475 / (27)) + (0.02404692124966882 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.15691096641052357 = (8.892419150683198 - 7.49710106802205) / 8.892419150683198 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.TwoClassVirtual.Call(testInput: pB = 0.50) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.78855414287074 < 9.981650069745994. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 36.59687086594768 (T) = (0 -8.86332145941383) / Math.Sqrt((0.018596014960292857 / (28)) + (0.03203257210750425 / (23))) is greater than 2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (23) - 2, .975) and 0.15772610334240594 = (10.523086960887971 - 8.86332145941383) / 10.523086960887971 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
151.75 ms 116.03 ms 0.76 0.27 True
497.63 μs 453.71 μs 0.91 0.24 False
524.78 μs 451.58 μs 0.86 0.15 True
423.35 μs 382.00 μs 0.90 0.14 True
1.24 ms 1.11 ms 0.90 0.27 False
1.19 ms 1.03 ms 0.86 0.30 False
12.63 ms 10.54 ms 0.83 0.61 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.LowLevelPerf.MeasureEvents #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 116.02799996428571 < 128.96178494352563. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 12.259304587964165 (T) = (0 -117741304.14818932) / Math.Sqrt((38184232782295.05 / (27)) + (19039365936326.3 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.13585032372954123 = (136251053.93355379 - 117741304.14818932) / 136251053.93355379 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.ClassVirtualMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 453.707006043956 < 472.70008875136506. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.35539979925984 (T) = (0 -456550.3141426736) / Math.Sqrt((1068218521.0947151 / (26)) + (108374946.65925886 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.08609449356706758 = (499559.64914210513 - 456550.3141426736) / 499559.64914210513 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 451.5768119747899 < 500.87297133804566. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 17.17764295934534 (T) = (0 -454035.0337743194) / Math.Sqrt((609738404.0810355 / (27)) + (78263930.31137694 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.16167837882672656 = (541600.0521838795 - 454035.0337743194) / 541600.0521838795 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.GenericClassGenericInstanceMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 382.0035582317074 < 405.4100701213602. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 12.039823628611828 (T) = (0 -384131.57214374933) / Math.Sqrt((256499921.45472324 / (26)) + (129580640.36899626 / (22))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (22) - 2, .975) and 0.11064462466266359 = (431921.3475243758 - 384131.57214374933) / 431921.3475243758 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.StaticDelegate #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1103241754807693 < 1.1185394823445538. IsChangePoint: Marked as a change because one of 8/24/2023 10:59:30 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.486940409520598 (T) = (0 -1086221.4773381595) / Math.Sqrt((9650950869.483799 / (29)) + (188691310.78366125 / (17))) is greater than 2.015367574442192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (17) - 2, .975) and 0.09970495119059251 = (1206517.2176328525 - 1086221.4773381595) / 1206517.2176328525 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.InstanceDelegate #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.0258559929687499 < 1.1107773762595663. IsChangePoint: Marked as a change because one of 8/24/2023 10:59:30 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.511322210287497 (T) = (0 -1028755.1330859122) / Math.Sqrt((19106315548.205173 / (31)) + (40084847.69732496 / (17))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (17) - 2, .975) and 0.17066739105897913 = (1240461.4529742596 - 1028755.1330859122) / 1240461.4529742596 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.539453122767856 < 11.262994292131696. IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.438819741962366 (T) = (0 -10236689.927024018) / Math.Sqrt((511064149574.144 / (27)) + (399799368292.53503 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.13702864191944206 = (11862143.31584853 - 10236689.927024018) / 11862143.31584853 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
298.69 ns 257.27 ns 0.86 0.25 False
300.58 ns 256.83 ns 0.85 0.38 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 257.27274925692035 < 283.5095354449171. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.708694258654354 (T) = (0 -260.9208129235803) / Math.Sqrt((211.95386905969718 / (27)) + (73.54401284009371 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.09985977953575091 = (289.8668529543207 - 260.9208129235803) / 289.8668529543207 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.ToUpper(s: "This is a much longer piece of text that might benefit more from vectorization.") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 256.83104079959384 < 288.5068519629159. IsChangePoint: Marked as a change because one of 8/22/2023 10:50:09 AM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.237857891133364 (T) = (0 -264.74988678264987) / Math.Sqrt((248.94815213296593 / (25)) + (59.4307357215089 / (23))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .975) and 0.09925688277466581 = (293.9238521168947 - 264.74988678264987) / 293.9238521168947 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
845.11 μs 775.29 μs 0.92 0.14 False
19.73 μs 18.46 μs 0.94 0.27 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CtorFromCollection<String>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 775.2922609126985 < 795.5836602604165. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.65937465835756 (T) = (0 -784020.8632566098) / Math.Sqrt((339073545.4283104 / (26)) + (299223336.581925 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.0648901730068428 = (838426.5041654267 - 784020.8632566098) / 838426.5041654267 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CtorFromCollection<String>.ConcurrentStack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.45666891820413 < 18.773751752488117. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.603066202006719 (T) = (0 -18588.450391162347) / Math.Sqrt((553540.9721988254 / (26)) + (214450.8389015454 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.07491226802273128 = (20093.71624832995 - 18588.450391162347) / 20093.71624832995 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
20.36 ns 17.32 ns 0.85 0.49 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.315268678413624 < 18.41898960810168. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 5.621118227199902 (T) = (0 -17.59714711585514) / Math.Sqrt((1.6804983534572129 / (26)) + (0.24814152455301278 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.0806516678414223 = (19.140891977840475 - 17.59714711585514) / 19.140891977840475 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare fd6d5fd0a330a8a6fd286a3d924f4e832aaeb479
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in GuardedDevirtualization.TwoClassInterface

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.89 ns 10.48 ns 0.88 0.16 True
13.02 ns 11.96 ns 0.92 0.31 False
15.43 ns 11.47 ns 0.74 0.24 False
14.93 ns 13.49 ns 0.90 0.31 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.TwoClassInterface*'
### Payloads [Baseline]() [Compare]() ### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.20) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.479958705357145 < 10.9750731936082. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.220968433577243 (T) = (0 -10.66370710809965) / Math.Sqrt((0.1380219510389478 / (26)) + (0.278488872121263 / (22))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (22) - 2, .975) and 0.09365824494068768 = (11.765657985603678 - 10.66370710809965) / 11.765657985603678 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.30) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.96071296438637 < 12.492951123719385. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.573393550828273 (T) = (0 -12.229537362555275) / Math.Sqrt((0.5462765767176632 / (27)) + (0.15056489806508075 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.10291349624113401 = (13.632506242500039 - 12.229537362555275) / 13.632506242500039 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.70) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.471661763622974 < 11.880029847788304. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.764930482960409 (T) = (0 -11.733225319568781) / Math.Sqrt((0.3856380495938737 / (26)) + (0.1976843496596167 / (22))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (22) - 2, .975) and 0.08171037460800856 = (12.777260022468624 - 11.733225319568781) / 12.777260022468624 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.TwoClassInterface.Call(testInput: pB = 0.40) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.492934427179964 < 14.273829741438087. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.046789498298589 (T) = (0 -13.783114392805272) / Math.Sqrt((0.302411345334471 / (27)) + (0.3071142891083051 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.08381338999094551 = (15.044003309182893 - 13.783114392805272) / 15.044003309182893 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
97.77 μs 88.44 μs 0.90 0.27 False
736.42 μs 668.25 μs 0.91 0.22 False
534.57 μs 481.02 μs 0.90 0.20 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CreateAddAndClear<Int32>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 88.43779583925189 < 95.20635882452913. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.721612904979462 (T) = (0 -92023.8464074321) / Math.Sqrt((12294777.183658363 / (26)) + (8600658.878588386 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.07168035658528397 = (99129.4831044758 - 92023.8464074321) / 99129.4831044758 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<Int32>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 668.2514046474358 < 701.8036203382554. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.6596824468868645 (T) = (0 -688302.8877987536) / Math.Sqrt((2462671392.2711368 / (25)) + (312157106.48017174 / (23))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .975) and 0.09291461593627513 = (758807.1640126865 - 688302.8877987536) / 758807.1640126865 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.CreateAddAndClear<Int32>.ImmutableList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 481.0185346320346 < 515.6706548594992. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 5.574637017418175 (T) = (0 -498617.0913209069) / Math.Sqrt((1832709630.2347295 / (26)) + (109733059.17831971 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.08841542349043906 = (546978.4199619762 - 498617.0913209069) / 546978.4199619762 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
148.80 μs 132.07 μs 0.89 0.17 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 132.0685381002825 < 132.67833180721894. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.37117867181255 (T) = (0 -128723.70868235207) / Math.Sqrt((39552256.73177816 / (21)) + (9075876.957310438 / (23))) is greater than 2.018081702816722 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (23) - 2, .975) and 0.09900130186274465 = (142867.8076322178 - 128723.70868235207) / 142867.8076322178 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in GuardedDevirtualization.ThreeClassInterface

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
14.28 ns 12.53 ns 0.88 0.18 True
13.87 ns 11.96 ns 0.86 0.27 False
19.60 ns 15.94 ns 0.81 0.23 False
13.60 ns 12.44 ns 0.91 0.12 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.ThreeClassInterface*'
### Payloads [Baseline]() [Compare]() ### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.67 pD=0.00) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.529011620835128 < 13.53580267741735. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 12.573246451058246 (T) = (0 -12.456935621328107) / Math.Sqrt((0.2571935370704786 / (26)) + (0.054866069762348174 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.10058891639134646 = (13.85010241518026 - 12.456935621328107) / 13.85010241518026 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.67 pD=0.33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.958870098415133 < 13.095066888724732. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.019664384714162 (T) = (0 -12.279242832903897) / Math.Sqrt((0.29905458504319105 / (27)) + (0.21700046120922342 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.08553503168140594 = (13.427789208242128 - 12.279242832903897) / 13.427789208242128 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.33 pD=0.33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.940686532586556 < 16.88930950548735. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.5453470387802 (T) = (0 -15.874993294850297) / Math.Sqrt((1.4249408434168096 / (27)) + (0.024164975041755632 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.11102089421198863 = (17.85755502181161 - 15.874993294850297) / 17.85755502181161 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.ThreeClassInterface.Call(testInput: pB=0.00 pD=0.33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.436236241652239 < 12.927934898651973. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 11.074470777110163 (T) = (0 -12.618869007965953) / Math.Sqrt((0.11299171259820445 / (26)) + (0.19749323390114923 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.09074628107582566 = (13.87827043797693 - 12.618869007965953) / 13.87827043797693 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
397.28 ns 372.84 ns 0.94 0.14 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_DateTime.ToString(format: "G") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 372.84174041569497 < 381.4079621609006. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.872701535427279 (T) = (0 -370.3442041414364) / Math.Sqrt((238.1039560012302 / (26)) + (80.93948616443984 / (22))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (22) - 2, .975) and 0.062344146397521075 = (394.96815672676917 - 370.3442041414364) / 394.96815672676917 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
29.41 μs 27.27 μs 0.93 0.27 False
9.18 μs 8.18 μs 0.89 0.35 False
13.52 μs 12.72 μs 0.94 0.39 False
17.58 μs 16.32 μs 0.93 0.29 False
21.79 μs 20.34 μs 0.93 0.22 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateForEach<String>.SortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.27085413804945 < 28.835416200117745. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 11.333294530075541 (T) = (0 -27127.135538621395) / Math.Sqrt((1312612.869217502 / (26)) + (593204.7724932577 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.10344828817733782 = (30257.190054852228 - 27127.135538621395) / 30257.190054852228 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<String>.Stack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.175500799616184 < 8.442574883656464. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.759529153319581 (T) = (0 -8242.809975822965) / Math.Sqrt((333167.3391988091 / (27)) + (41737.130559367644 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.08889029067426565 = (9047.000478046763 - 8242.809975822965) / 9047.000478046763 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<String>.ConcurrentStack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.718155739071195 < 12.750565404240303. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.191170692405262 (T) = (0 -12865.04702033333) / Math.Sqrt((632930.91292073 / (26)) + (164210.6020829197 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.07867102214276545 = (13963.575801397235 - 12865.04702033333) / 13963.575801397235 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<String>.ConcurrentBag(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 16.32236501384562 < 16.91973460808674. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.866555198228957 (T) = (0 -16386.294288657788) / Math.Sqrt((333017.4795139723 / (27)) + (138273.21692349148 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.08241852037266136 = (17858.13538358776 - 16386.294288657788) / 17858.13538358776 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEach<String>.ImmutableQueue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.340063762626265 < 21.061624969896982. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.121966538917857 (T) = (0 -20494.37013612183) / Math.Sqrt((2801899.0477310796 / (27)) + (119700.89340194459 / (21))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (21) - 2, .975) and 0.10312264994529126 = (22850.805781717747 - 20494.37013612183) / 22850.805781717747 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.TryAddGiventSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
99.63 μs 92.10 μs 0.92 0.19 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.TryAddGiventSize<String>.ConcurrentDictionary(Count: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 92.0973287990196 < 94.05603239143811. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.707513656753216 (T) = (0 -93560.24381835418) / Math.Sqrt((16455713.972064454 / (27)) + (3957507.7328982754 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.05960144516501099 = (99489.9910653001 - 93560.24381835418) / 99489.9910653001 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.84 secs 3.39 secs 0.88 0.31 False
2.44 secs 2.02 secs 0.83 0.40 False
1.40 secs 1.28 secs 0.92 0.29 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'
### Payloads [Baseline]() [Compare]() ### LinqBenchmarks.Count00LinqMethodX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.3931486924615384 < 3.51628851586717. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.3233916540736 (T) = (0 -3405627852.414397) / Math.Sqrt((34988967428334796 / (26)) + (7946420462806492 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.09132937032013115 = (3747923330.1666455 - 3405627852.414397) / 3747923330.1666455 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### LinqBenchmarks.Where00LinqQueryX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.017222923153846 < 2.1819050825461535. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.346465042446438 (T) = (0 -2081059567.4581811) / Math.Sqrt((20276507417094584 / (26)) + (5012844610311786 / (24))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (24) - 2, .975) and 0.13519629821947804 = (2406395304.7073474 - 2081059567.4581811) / 2406395304.7073474 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### LinqBenchmarks.Count00ForX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2833209231538463 < 1.3320852556604168. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 5.549097486019369 (T) = (0 -1316998687.712868) / Math.Sqrt((3364974941658937 / (26)) + (4476161304643516.5 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.07049926693912369 = (1416888272.240462 - 1316998687.712868) / 1416888272.240462 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.Tests.DictionarySequentialKeys

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
37.39 ns 31.69 ns 0.85 0.50 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Tests.DictionarySequentialKeys.ContainsKey_3k_Int_32ByteRefsValue #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 31.688561529108593 < 35.25020233806674. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.26243925819571 (T) = (0 -32.96802704537409) / Math.Sqrt((7.689442495609564 / (27)) + (2.47252459791658 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.10632528516652613 = (36.890410457139666 - 32.96802704537409) / 36.890410457139666 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.78 μs 1.67 μs 0.94 0.12 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.66881690447772 < 1.695745184902543. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.180526315472687 (T) = (0 -1648.6696809082493) / Math.Sqrt((2424.0434441996085 / (27)) + (1752.310166112917 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.0669354531157141 = (1766.9406542275465 - 1648.6696809082493) / 1766.9406542275465 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in GuardedDevirtualization.ThreeClassVirtual

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.10 ns 7.79 ns 0.86 0.10 True
9.40 ns 7.93 ns 0.84 0.27 True
8.82 ns 7.40 ns 0.84 0.10 True
9.41 ns 8.18 ns 0.87 0.18 True
8.78 ns 7.54 ns 0.86 0.12 True
12.05 ns 10.04 ns 0.83 0.29 True
9.70 ns 7.52 ns 0.78 0.28 True

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'GuardedDevirtualization.ThreeClassVirtual*'
### Payloads [Baseline]() [Compare]() ### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.67 pD=0.33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.787662680709557 < 8.50964737891978. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 22.003415632244078 (T) = (0 -7.605691452610095) / Math.Sqrt((0.04731033497674864 / (26)) + (0.0429945911008453 / (20))) is greater than 2.015367574442192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (20) - 2, .975) and 0.15416841976259435 = (8.99196912283099 - 7.605691452610095) / 8.99196912283099 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.33 pD=0.67) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.933898609691705 < 8.968486293907995. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 17.939333904334063 (T) = (0 -8.061790367177226) / Math.Sqrt((0.1555197858658552 / (27)) + (0.050246966065684434 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.16551282966004544 = (9.660772092988562 - 8.061790367177226) / 9.660772092988562 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.00 pD=0.67) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.396653767537495 < 8.64628717427389. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 28.78925775355872 (T) = (0 -7.373498833682478) / Math.Sqrt((0.05406078852835275 / (27)) + (0.031084469473965547 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.1844133335916755 = (9.040729989068907 - 7.373498833682478) / 9.040729989068907 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.00 pD=0.33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.18338021243503 < 8.935390819214371. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 19.44818107565163 (T) = (0 -8.094439457367718) / Math.Sqrt((0.10824212971257108 / (26)) + (0.039263018414683604 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.15546663386394732 = (9.584511141817599 - 8.094439457367718) / 9.584511141817599 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.67 pD=0.00) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.535202439266707 < 8.374914497616343. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 27.207056271009137 (T) = (0 -7.504795576363158) / Math.Sqrt((0.05645058807383562 / (27)) + (0.01072870162951517 / (21))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (21) - 2, .975) and 0.15605616338240721 = (8.892529633774345 - 7.504795576363158) / 8.892529633774345 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.33 pD=0.33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.040449935107073 < 11.683091348711448. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 17.533405513493097 (T) = (0 -10.157400966180262) / Math.Sqrt((0.2506091024220688 / (27)) + (0.09836828435649166 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.16735946042974903 = (12.199022847749859 - 10.157400966180262) / 12.199022847749859 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### GuardedDevirtualization.ThreeClassVirtual.Call(testInput: pB=0.33 pD=0.00) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.524697637477826 < 9.173472634720827. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 17.131591661298923 (T) = (0 -7.8243989720580105) / Math.Sqrt((0.18291453252666276 / (25)) + (0.11376214197757308 / (23))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .975) and 0.1951453919735054 = (9.721506088215678 - 7.8243989720580105) / 9.721506088215678 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Linq.Tests.Perf_OrderBy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
779.59 μs 710.63 μs 0.91 0.13 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_OrderBy*'
### Payloads [Baseline]() [Compare]() ### System.Linq.Tests.Perf_OrderBy.OrderByString(NumberOfPeople: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 710.6306846590909 < 718.7330821935232. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.881044473819351 (T) = (0 -704635.8276827221) / Math.Sqrt((339494017.1207359 / (27)) + (185277855.06025666 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.05414605516552775 = (744973.1869607372 - 704635.8276827221) / 744973.1869607372 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.97 ns 2.88 ns 0.36 0.82 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Int64>.AbsBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.882329401909665 < 7.565408659594935. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.17296823860804 (T) = (0 -4.078407117966976) / Math.Sqrt((2.4961024053535503 / (26)) + (3.61563197627672 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.5020870831745576 = (8.191004852755766 - 4.078407117966976) / 8.191004852755766 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
14.15 μs 12.82 μs 0.91 0.20 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Guid, Guid&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 1000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.819373115161556 < 13.33302722133045. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 4.918586188388613 (T) = (0 -13300.337166524814) / Math.Sqrt((229120.885517159 / (21)) + (266450.7116377356 / (23))) is greater than 2.018081702816722 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (23) - 2, .975) and 0.05255085039929503 = (14038.048556094158 - 13300.337166524814) / 14038.048556094158 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.58 μs 4.90 μs 0.88 0.30 False
11.28 μs 9.99 μs 0.89 0.38 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateFor<Int32>.List(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.899460156577567 < 5.210770410497184. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.68684810591104 (T) = (0 -4935.249568593447) / Math.Sqrt((98210.22013349878 / (27)) + (524.6631801641076 / (21))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (21) - 2, .975) and 0.11586256907690157 = (5581.993699147787 - 4935.249568593447) / 5581.993699147787 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateFor<Int32>.IList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.989227318895892 < 10.613260619991339. IsChangePoint: Marked as a change because one of 8/22/2023 3:00:25 AM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.4721297937626785 (T) = (0 -10318.596042892632) / Math.Sqrt((517769.98273545335 / (23)) + (45874.359393378865 / (23))) is greater than 2.015367574442192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (23) - 2, .975) and 0.10181847160714878 = (11488.31913895632 - 10318.596042892632) / 11488.31913895632 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.IterateForEachNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
13.78 μs 11.85 μs 0.86 0.18 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateForEachNonGeneric<String>.Stack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.848593199924126 < 12.148742829922568. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.992913908450543 (T) = (0 -11948.880950971286) / Math.Sqrt((152178.24640476235 / (25)) + (71326.9760104444 / (23))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .975) and 0.08104007133743077 = (13002.613692156721 - 11948.880950971286) / 13002.613692156721 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
78.60 ms 72.08 ms 0.92 0.08 True
79.84 ms 73.80 ms 0.92 0.04 True
80.72 ms 74.66 ms 0.92 0.14 True
79.44 ms 72.79 ms 0.92 0.12 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: AllEscaped) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 72.08269644642857 < 75.22695903616072. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 16.433853696756422 (T) = (0 -72197881.1148491) / Math.Sqrt((3940429510038.5835 / (27)) + (183759288087.98193 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.08198394378911189 = (78645553.77478462 - 72197881.1148491) / 78645553.77478462 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: False, Escaped: AllEscaped) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 73.80205001666667 < 77.06260694440246. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 36.8630990597574 (T) = (0 -73923085.58428656) / Math.Sqrt((877377109434.3384 / (26)) + (225674705049.77173 / (22))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (22) - 2, .975) and 0.09469930919645886 = (81655836.9337736 - 73923085.58428656) / 81655836.9337736 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: True, Escaped: AllEscaped) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 74.66103328333332 < 76.62453746830927. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.711947268678225 (T) = (0 -74803889.74481803) / Math.Sqrt((1970435149112.4355 / (26)) + (11921509332196.195 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.0823741639435539 = (81518944.6564543 - 74803889.74481803) / 81518944.6564543 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: False, Escaped: AllEscaped) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 72.79125007692308 < 75.4712034141369. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 11.211433000035791 (T) = (0 -72346708.62571388) / Math.Sqrt((13229054571189.965 / (27)) + (408452890561.9255 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.09951273243025134 = (80341734.11575767 - 72346708.62571388) / 80341734.11575767 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
70.19 μs 60.14 μs 0.86 0.29 False
782.73 μs 722.84 μs 0.92 0.13 False
30.23 μs 27.91 μs 0.92 0.35 False
8.98 μs 8.52 μs 0.95 0.47 False
9.20 μs 8.33 μs 0.91 0.62 False
15.59 μs 11.92 μs 0.76 0.43 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.AddGivenSize<String>.ObservableCollection(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 60.13888330152671 < 63.632737272196316. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.211564457750418 (T) = (0 -61274.707221536744) / Math.Sqrt((4725588.155766502 / (27)) + (3408854.1726529095 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.08654753912261093 = (67080.34609998333 - 61274.707221536744) / 67080.34609998333 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.AddGivenSize<String>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 722.8409058441558 < 727.4066766707685. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.986630260296469 (T) = (0 -703822.5786523108) / Math.Sqrt((318573952.52787995 / (27)) + (268629976.60123825 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.07031880087778206 = (757057.9886060326 - 703822.5786523108) / 757057.9886060326 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.AddGivenSize<String>.HashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 27.91011058242247 < 28.408995592951612. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.21337385848606 (T) = (0 -27203.27532087225) / Math.Sqrt((1287341.8131296267 / (26)) + (558403.0253902004 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.08425138554747397 = (29706.05130222942 - 27203.27532087225) / 29706.05130222942 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.AddGivenSize<String>.Stack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.517549679487178 < 9.52875117882336. IsChangePoint: Marked as a change because one of 8/22/2023 10:50:09 AM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.912281522904888 (T) = (0 -8694.10590053473) / Math.Sqrt((639866.8303274976 / (24)) + (140342.96340603588 / (25))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (25) - 2, .975) and 0.15551973293949004 = (10295.214985659064 - 8694.10590053473) / 10295.214985659064 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.AddGivenSize<String>.List(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.330542472299168 < 8.606491954181626. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 5.051280728181854 (T) = (0 -8422.98162278202) / Math.Sqrt((506407.0742133434 / (25)) + (239708.67552402607 / (23))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .975) and 0.09505484525462779 = (9307.726085512912 - 8422.98162278202) / 9307.726085512912 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.AddGivenSize<String>.ICollection(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.92106828716364 < 13.26633984074502. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.970870516624009 (T) = (0 -12184.27765573822) / Math.Sqrt((1123388.7575370236 / (26)) + (220966.77073085486 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.13069390187165253 = (14016.095920610107 - 12184.27765573822) / 14016.095920610107 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_FromStream<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
20.14 μs 18.20 μs 0.90 0.10 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_FromStream<SimpleStructWithProperties>.XmlSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.203509266241298 < 18.70711968246531. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.581432496700964 (T) = (0 -18422.196020361018) / Math.Sqrt((262174.0687679508 / (26)) + (110121.2201159611 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.06546032492635569 = (19712.588466518875 - 18422.196020361018) / 19712.588466518875 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Tests.Perf_HashCode

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
96.84 μs 90.88 μs 0.94 0.25 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_HashCode*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_HashCode.Combine_4 #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 90.87676479443522 < 91.99287648920351. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.676728974265455 (T) = (0 -90863.22388505515) / Math.Sqrt((22420717.087668195 / (28)) + (30298.211998192 / (23))) is greater than 2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (23) - 2, .975) and 0.08707220434341846 = (99529.4746390167 - 90863.22388505515) / 99529.4746390167 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in PerfLabTests.DelegatePerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
132.62 ms 116.97 ms 0.88 0.27 True
13.24 ms 11.76 ms 0.89 0.28 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.DelegatePerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.DelegatePerf.MulticastDelegateInvoke(length: 1000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 116.96960003333334 < 129.9742816825. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 15.577042419914537 (T) = (0 -118708220.849098) / Math.Sqrt((26929092118538.523 / (26)) + (9314052877729.229 / (20))) is greater than 2.015367574442192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (20) - 2, .975) and 0.13851738246810574 = (137795259.51340872 - 118708220.849098) / 137795259.51340872 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.DelegatePerf.MulticastDelegateInvoke(length: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.75909821875 < 12.538338364330357. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.068308665261362 (T) = (0 -12011507.152006505) / Math.Sqrt((1360104546290.5168 / (27)) + (339176201358.20605 / (21))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (21) - 2, .975) and 0.14766841812550305 = (14092528.550438203 - 12011507.152006505) / 14092528.550438203 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.83 μs 13.77 μs 0.73 0.43 False
40.20 μs 37.30 μs 0.93 0.41 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IndexerSet<String>.IList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.770519599613348 < 15.78900447279088. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.109921110583223 (T) = (0 -13961.937487213534) / Math.Sqrt((1647756.6056033415 / (26)) + (59715.37867452828 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.12982438531246893 = (16044.96523638747 - 13961.937487213534) / 16044.96523638747 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IndexerSet<String>.Dictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 37.301877983176844 < 38.48900849625917. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 5.0575165587547835 (T) = (0 -37933.90202912549) / Math.Sqrt((4845119.351338839 / (26)) + (1457329.38407914 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.06246244096647963 = (40461.20783494842 - 37933.90202912549) / 40461.20783494842 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
600.83 ns 568.39 ns 0.95 0.18 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToWriter(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 568.3850904818709 < 571.1782540047332. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.460516030731161 (T) = (0 -563.9128727495906) / Math.Sqrt((469.2063460360911 / (26)) + (123.2936555745726 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.060156843834737746 = (600.0074257607639 - 563.9128727495906) / 600.0074257607639 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.Concurrent.IsEmpty<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
115.29 ns 101.17 ns 0.88 0.43 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Concurrent.IsEmpty<Int32>.Dictionary(Size: 0) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 101.1722339700579 < 110.38780773812637. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.3499661323507555 (T) = (0 -105.49126664387738) / Math.Sqrt((39.12556656184228 / (26)) + (23.008127281668298 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.09932460446528486 = (117.12462355125075 - 105.49126664387738) / 117.12462355125075 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare fd6d5fd0a330a8a6fd286a3d924f4e832aaeb479
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<Hashtable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
110.58 μs 102.14 μs 0.92 0.08 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Hashtable&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromString(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 102.14498721988795 < 104.51248723617951. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.303260408220384 (T) = (0 -103462.69677688793) / Math.Sqrt((9966061.511091147 / (26)) + (1644122.1031052165 / (22))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (22) - 2, .975) and 0.0573651264090766 = (109759.03785815989 - 103462.69677688793) / 109759.03785815989 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.39 μs 5.08 μs 0.94 0.08 False
95.02 μs 88.03 μs 0.93 0.27 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.AddGivenSize<Int32>.Queue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.0822508689621415 < 5.120987997702394. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.479606735812482 (T) = (0 -5129.760078071228) / Math.Sqrt((15366.88004110968 / (27)) + (7453.042729231541 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.05754158512298118 = (5442.956418125472 - 5129.760078071228) / 5442.956418125472 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.AddGivenSize<Int32>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 88.0321563559322 < 89.34121374084597. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.0898475472463725 (T) = (0 -86895.57646448634) / Math.Sqrt((12905196.152553512 / (27)) + (4579717.19761772 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.06330575506614006 = (92768.34669847047 - 86895.57646448634) / 92768.34669847047 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.68 ns 2.89 ns 0.38 0.80 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.AbsBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.8873483489341463 < 7.490064025066162. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 13.681120434729676 (T) = (0 -3.9052108364245726) / Math.Sqrt((1.5015760677118315 / (26)) + (1.3005512451582442 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.542207985390752 = (8.530535072259607 - 3.9052108364245726) / 8.530535072259607 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
71.22 μs 64.45 μs 0.90 0.15 True
84.03 μs 66.22 μs 0.79 0.19 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Sort<Int32>.Array_ComparerStruct(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 64.44912 < 68.31103434119048. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 13.873516443312933 (T) = (0 -65608.16721875458) / Math.Sqrt((3166758.298512218 / (28)) + (4182537.253806947 / (23))) is greater than 2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (23) - 2, .975) and 0.10301203342211898 = (73142.75069827054 - 65608.16721875458) / 73142.75069827054 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Sort<Int32>.Array_ComparerClass(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 66.21627330666666 < 69.22247651566667. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.61035454904633 (T) = (0 -65386.84791270266) / Math.Sqrt((20825994.411506336 / (27)) + (490250.0369151931 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.1157130881132073 = (73943.02350714148 - 65386.84791270266) / 73943.02350714148 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.41 μs 4.97 μs 0.92 0.27 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateFor<String>.List(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.973734062538338 < 5.346286980440636. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.578160838318283 (T) = (0 -5019.723767482928) / Math.Sqrt((51575.938151056034 / (27)) + (34380.31606133457 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.10018758001536343 = (5578.633564058424 - 5019.723767482928) / 5578.633564058424 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.82 ms 2.47 ms 0.88 0.46 False
633.85 μs 589.02 μs 0.93 0.17 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrue<String>.Array(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.465929256181319 < 2.6650010256249996. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 5.870100088796493 (T) = (0 -2593934.3390945173) / Math.Sqrt((27981079467.795864 / (27)) + (25420547225.05522 / (21))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (21) - 2, .975) and 0.09687673406370463 = (2872181.945623233 - 2593934.3390945173) / 2872181.945623233 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrue<String>.SortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 589.0209875 < 600.103373492857. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 10.340247009009477 (T) = (0 -584140.3450355694) / Math.Sqrt((211929588.5964482 / (27)) + (321902230.938956 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.07641303707594983 = (632469.2405641994 - 584140.3450355694) / 632469.2405641994 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.95 μs 8.08 μs 0.90 0.15 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (pl-PL, None, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.076162155950524 < 8.389922582211831. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.259597657773035 (T) = (0 -8139.186739237691) / Math.Sqrt((132471.27752650494 / (27)) + (7463.313108561397 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.06837605684159852 = (8736.558134867308 - 8139.186739237691) / 8736.558134867308 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
126.20 μs 114.22 μs 0.91 0.15 True
115.98 μs 107.71 μs 0.93 0.15 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 114.21793541449425 < 122.32804218841483. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 14.683006042039223 (T) = (0 -118378.94407629858) / Math.Sqrt((5927993.714623487 / (21)) + (8485588.94337491 / (23))) is greater than 2.018081702816722 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (23) - 2, .975) and 0.09098633048441156 = (130227.90310665237 - 118378.94407629858) / 130227.90310665237 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsKeyTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 107.71227327056626 < 111.86840903161837. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 11.04590418747185 (T) = (0 -107944.54400685061) / Math.Sqrt((21067553.721911896 / (21)) + (3296202.7490086164 / (23))) is greater than 2.018081702816722 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (23) - 2, .975) and 0.09875029959283356 = (119772.07199967273 - 107944.54400685061) / 119772.07199967273 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.IterateForEachNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
27.86 μs 22.76 μs 0.82 0.25 False
12.81 μs 11.85 μs 0.93 0.35 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateForEachNonGeneric<Int32>.SortedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.76181569994786 < 22.871810681598138. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.882047866924102 (T) = (0 -22724.94143282907) / Math.Sqrt((2665789.9925008444 / (27)) + (59005.167992842384 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.08791288640671253 = (24915.31904589811 - 22724.94143282907) / 24915.31904589811 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.IterateForEachNonGeneric<Int32>.Stack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.846620132947036 < 12.33180456898816. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.793843658072691 (T) = (0 -12096.18156574749) / Math.Sqrt((570127.6778413745 / (27)) + (350848.3911877633 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.09674931622775033 = (13391.83217136372 - 12096.18156574749) / 13391.83217136372 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.ComponentModel.Tests.Perf_TypeDescriptorTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.06 μs 950.76 ns 0.89 0.13 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'
### Payloads [Baseline]() [Compare]() ### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(int)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 950.7587376036523 < 0.9817915261394382. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 12.757363708718657 (T) = (0 -954.0632253152914) / Math.Sqrt((804.873120790751 / (28)) + (446.08625612778155 / (23))) is greater than 2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (23) - 2, .975) and 0.08490000882194079 = (1042.5781166133252 - 954.0632253152914) / 1042.5781166133252 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.91 μs 1.75 μs 0.92 0.21 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'
### Payloads [Baseline]() [Compare]() ### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.7491993109635056 < 1.7981221977465918. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.064729330788428 (T) = (0 -1769.6243186083004) / Math.Sqrt((7846.138901869775 / (27)) + (1568.9292992804612 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.07946611987366427 = (1922.389123109118 - 1769.6243186083004) / 1922.389123109118 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Xml.Linq.Perf_XName

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
159.03 ns 139.46 ns 0.88 0.27 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XName*'
### Payloads [Baseline]() [Compare]() ### System.Xml.Linq.Perf_XName.CreateElement #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 139.46207744620173 < 148.42145775997017. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.597449254610025 (T) = (0 -143.3677479811843) / Math.Sqrt((55.56389688039215 / (26)) + (14.668771557434047 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.08111427028923149 = (156.02347859544102 - 143.3677479811843) / 156.02347859544102 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.56 μs 2.40 μs 0.94 0.06 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeObjectProperty(Mode: SourceGen) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.3985835675752414 < 2.429832438480968. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 15.82339240739519 (T) = (0 -2403.529641374869) / Math.Sqrt((2315.148792422462 / (27)) + (1055.5582949409452 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.07074461875038267 = (2586.5114045858086 - 2403.529641374869) / 2586.5114045858086 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.ContainsKeyFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
123.62 μs 110.24 μs 0.89 0.10 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsKeyFalse<String, String>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 110.23879110915493 < 117.60911647650413. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 11.53318697001873 (T) = (0 -114315.8257149442) / Math.Sqrt((6551539.036247224 / (20)) + (7050744.959096199 / (20))) is greater than 2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (20) - 2, .975) and 0.0768111792946491 = (123827.13389835313 - 114315.8257149442) / 123827.13389835313 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
113.46 μs 102.25 μs 0.90 0.18 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 102.24921689471243 < 108.98136527545219. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 15.35107709181645 (T) = (0 -103117.50845280461) / Math.Sqrt((5637532.668656432 / (20)) + (5114636.7191456435 / (23))) is greater than 2.019540970439574 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (23) - 2, .975) and 0.09560656583906088 = (114018.41782329281 - 103117.50845280461) / 114018.41782329281 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.ContainsKeyTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
659.15 μs 611.58 μs 0.93 0.22 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsKeyTrue<String, String>.ImmutableSortedDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 611.5791548763738 < 627.7258477052569. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.505392522054564 (T) = (0 -605983.0501353825) / Math.Sqrt((339971424.9097513 / (21)) + (430875421.27396923 / (23))) is greater than 2.018081702816722 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (23) - 2, .975) and 0.07659180756538286 = (656246.1272275205 - 605983.0501353825) / 656246.1272275205 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
64.86 ns 58.41 ns 0.90 0.51 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GetHashCodeBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 58.41386791078703 < 60.335509955766824. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.087356292657587 (T) = (0 -57.180219929178364) / Math.Sqrt((16.2896681660841 / (27)) + (6.901155650351567 / (22))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (22) - 2, .975) and 0.1060997857020768 = (63.967117374603376 - 57.180219929178364) / 63.967117374603376 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
142.11 μs 125.96 μs 0.89 0.29 False
122.11 μs 115.02 μs 0.94 0.08 True
688.44 μs 614.99 μs 0.89 0.12 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrue<Int32>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 125.95820139688942 < 134.31392177932568. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.576993235689832 (T) = (0 -128352.34714314993) / Math.Sqrt((53946824.73573002 / (27)) + (65299450.86828797 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.10128387591590912 = (142817.45225608113 - 128352.34714314993) / 142817.45225608113 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrue<Int32>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 115.0150674238445 < 117.07586636541829. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 13.91634849677128 (T) = (0 -112676.64379160188) / Math.Sqrt((7830672.241380274 / (25)) + (6390983.003774163 / (23))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .975) and 0.08672090765902785 = (123375.91513540762 - 112676.64379160188) / 123375.91513540762 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrue<Int32>.LinkedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 614.9897429487179 < 653.7443880852843. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 13.812558038207793 (T) = (0 -612877.2030716516) / Math.Sqrt((349459097.89184105 / (27)) + (533372908.1612478 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.11930981004458427 = (695905.5636837264 - 612877.2030716516) / 695905.5636837264 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.ContainsTrueComparer<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
68.67 μs 57.60 μs 0.84 0.38 False
164.06 μs 146.41 μs 0.89 0.25 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrueComparer<String>.HashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 57.595969217995176 < 65.76701183347919. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 6.404143711383128 (T) = (0 -63132.75069209785) / Math.Sqrt((7694187.618376831 / (26)) + (12251970.89087012 / (22))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (22) - 2, .975) and 0.0856544624348058 = (69046.9282107656 - 63132.75069209785) / 69046.9282107656 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<String>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 146.40811020171958 < 159.0334284649311. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 8.063745410802467 (T) = (0 -154319.54595590293) / Math.Sqrt((30064228.877106424 / (27)) + (22467242.06760801 / (24))) is greater than 2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (24) - 2, .975) and 0.0696020193026154 = (165864.01642900376 - 154319.54595590293) / 165864.01642900376 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
133.16 μs 117.84 μs 0.88 0.13 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.TryGetValueTrue<Int32, Int32>.ImmutableDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 117.8436781716418 < 125.9755402920244. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 14.57275786962918 (T) = (0 -119213.8747020741) / Math.Sqrt((11546592.104672063 / (21)) + (5146152.876316133 / (23))) is greater than 2.018081702816722 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (23) - 2, .975) and 0.09707758782565673 = (132031.1392149333 - 119213.8747020741) / 132031.1392149333 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
190.32 ns 166.79 ns 0.88 0.31 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Byte>.GetHashCodeBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 166.78629935714056 < 175.95822945805912. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 7.7813339557601315 (T) = (0 -167.61325716858835) / Math.Sqrt((60.70948601025912 / (27)) + (22.74405429650841 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.0770905337902723 = (181.61397548229164 - 167.61325716858835) / 181.61397548229164 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.Tests.Perf_SortedSet

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
39.11 μs 35.86 μs 0.92 0.08 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_SortedSet*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Tests.Perf_SortedSet.EnumerateViewBetween #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 35.85806328125 < 36.81859683912081. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.68647487869354 (T) = (0 -35920.51593287477) / Math.Sqrt((596826.7693172763 / (27)) + (552236.1376556702 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.05473888073086374 = (38000.62776373162 - 35920.51593287477) / 38000.62776373162 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.25 μs 2.77 μs 0.85 0.18 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>.DataContractSerializer_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.772085969939363 < 2.7955998060029286. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.7818076840281 (T) = (0 -2689.804259613167) / Math.Sqrt((13278.422607976741 / (26)) + (3277.5102564327717 / (22))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (22) - 2, .975) and 0.0854252723773295 = (2941.043720511496 - 2689.804259613167) / 2941.043720511496 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
29.67 μs 26.91 μs 0.91 0.08 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 26.910976883032625 < 27.712047231884277. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 9.251147110645654 (T) = (0 -27397.73851229371) / Math.Sqrt((391928.78271282197 / (26)) + (242292.8239181509 / (23))) is greater than 2.0117405137283924 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .975) and 0.051264799811786436 = (28878.172230627097 - 27397.73851229371) / 28878.172230627097 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.21 μs 10.00 μs 0.89 0.33 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateForEach<Int32>.FrozenDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.004088995176845 < 10.200229194329532. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 5.410077079776081 (T) = (0 -10218.586973298972) / Math.Sqrt((346565.9730036921 / (27)) + (170179.59892016381 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.07003717447033964 = (10988.1671533257 - 10218.586973298972) / 10988.1671533257 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 3a9412266ac60dc5e6156909ab2ddb9e89eb0df8
Compare e3368e865281f97585316af5702e7bac5d4c1a3a
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Improvements in XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
821.97 ns 751.67 ns 0.91 0.21 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList*'
### Payloads [Baseline]() [Compare]() ### XmlDocumentTests.XmlNodeListTests.Perf_XmlNodeList.Enumerator #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 751.6698829381555 < 791.6420926793642. IsChangePoint: Marked as a change because one of 8/22/2023 10:44:27 PM, 8/30/2023 5:40:03 PM falls between 8/22/2023 3:00:25 AM and 8/30/2023 5:40:03 PM. IsImprovementStdDev: Marked as improvement because 12.280208968677192 (T) = (0 -760.5579662975299) / Math.Sqrt((1081.8606064972048 / (27)) + (397.0723457580138 / (23))) is greater than 2.0106347576230403 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (23) - 2, .975) and 0.10893889803094974 = (853.5418778991284 - 760.5579662975299) / 853.5418778991284 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)