search

DrewScoggins / performance-2

This repo contains benchmarks used for testing the performance of all .NET Runtimes
MIT License
0 stars 0 forks source link

[Perf] Windows/x86: 12 Improvements on 7/12/2023 9:48:15 PM #8866

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare d9f6b98de38081b71fa9b5a463f302288546435d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[ImmutableList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.CreateAddAndClear(Int32).ImmutableList(Size%3a%20512).html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 80.59 μs 71.01 μs 0.88 0.04 False 595810.688405797 553946.3141025641 0.9297354426197878)

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.CreateAddAndClear<Int32>.ImmutableList(Size: 512) #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-14febf6915b34d88ad/Collect%20System.Collections.CreateAddAndClear_Int32_.ImmutableList_Size%3A%20512_/1/artifacts/BenchmarkDotNet.Artifacts/System.Collections.CreateAddAndClear_Int32_.ImmutableList%28Size_%20512%29-20230720-125630.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A03Z&sr=c&sp=rl&sig=76JYIqBigqtrevT9ojqnwOb1rsKs2nDXHUgaGqQ%2FOoQ%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-b6883d69a090497a86/Collect%20System.Collections.CreateAddAndClear_Int32_.ImmutableList_Size%3A%20512_/1/artifacts/BenchmarkDotNet.Artifacts/System.Collections.CreateAddAndClear_Int32_.ImmutableList%28Size_%20512%29-20230720-125632.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A02Z&sr=c&sp=rl&sig=qytlrPyUhJpOekEMO4jKzhV%2F%2BmzOeeMr%2FhQ4w25VhCI%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 71.00914772727272 < 76.85069754906819. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 54.58650049977566 (T) = (0 -71647.02102187574) / Math.Sqrt((8589903.93132545 / (299)) + (101449.81532643845 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.13673554516266828 = (82995.44898483799 - 71647.02102187574) / 82995.44898483799 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/System_Collections_CreateAddAndClear_Int32__ImmutableList_Size__512__baseline_fd595ae8-2475-489e-b18a-31d0e2a8e47e.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/System_Collections_CreateAddAndClear_Int32__ImmutableList_Size__512__compare_fd595ae8-2475-489e-b18a-31d0e2a8e47e.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FSystem_Collections_CreateAddAndClear_Int32__ImmutableList_Size__512__baseline_fd595ae8-2475-489e-b18a-31d0e2a8e47e.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FSystem_Collections_CreateAddAndClear_Int32__ImmutableList_Size__512__compare_fd595ae8-2475-489e-b18a-31d0e2a8e47e.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare d9f6b98de38081b71fa9b5a463f302288546435d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Devirtualization.EqualityComparer

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[ValueTupleCompare - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/Devirtualization.EqualityComparer.ValueTupleCompare.html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 5.70 ns 3.34 ns 0.59 0.16 False 49.35068483623049 33.08733019026835 0.6704533138712898)
[ValueTupleCompareNoOpt - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/Devirtualization.EqualityComparer.ValueTupleCompareNoOpt.html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 9.42 ns 7.53 ns 0.80 0.18 False 60.79756141350347 49.80025165276216 0.8191159397669731)
[ValueTupleCompareWrapped - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/Devirtualization.EqualityComparer.ValueTupleCompareWrapped.html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 5.73 ns 4.27 ns 0.75 0.16 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Devirtualization.EqualityComparer*'
### Payloads [Baseline]() [Compare]() ### Devirtualization.EqualityComparer.ValueTupleCompare #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-97a119928ea44c09b7/Collect%20Devirtualization.EqualityComparer.ValueTupleCompare/1/artifacts/BenchmarkDotNet.Artifacts/Devirtualization.EqualityComparer.ValueTupleCompare-20230720-125636.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A03Z&sr=c&sp=rl&sig=71gBi4N79oKkyQSVBLyM74N9KSAoMCkeny102379SSc%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-3dce8c578ae1477ba6/Collect%20Devirtualization.EqualityComparer.ValueTupleCompare/1/artifacts/BenchmarkDotNet.Artifacts/Devirtualization.EqualityComparer.ValueTupleCompare-20230720-125636.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A03Z&sr=c&sp=rl&sig=syfce%2Bu8T6WwM0IvVK6vdmnmkkCz71FiUfc5yrTiElc%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.336529863631574 < 5.421784488553459. IsChangePoint: Marked as a change because one of 7/1/2023 6:36:20 PM, 7/5/2023 3:16:33 PM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 69.79425417047419 (T) = (0 -3.3444764052327187) / Math.Sqrt((0.18539390260906588 / (299)) + (0.00117094621143151 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.36617721403570797 = (5.276674299653749 - 3.3444764052327187) / 5.276674299653749 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Devirtualization_EqualityComparer_ValueTupleCompare_baseline_27a33742-6b9e-48cf-8c60-087c54e0ea38.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Devirtualization_EqualityComparer_ValueTupleCompare_compare_27a33742-6b9e-48cf-8c60-087c54e0ea38.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FDevirtualization_EqualityComparer_ValueTupleCompare_baseline_27a33742-6b9e-48cf-8c60-087c54e0ea38.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FDevirtualization_EqualityComparer_ValueTupleCompare_compare_27a33742-6b9e-48cf-8c60-087c54e0ea38.log) ### Devirtualization.EqualityComparer.ValueTupleCompareNoOpt #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-42836ebb3f77474fa1/Collect%20Devirtualization.EqualityComparer.ValueTupleCompareNoOpt/1/artifacts/BenchmarkDotNet.Artifacts/Devirtualization.EqualityComparer.ValueTupleCompareNoOpt-20230720-125639.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A04Z&sr=c&sp=rl&sig=xtg5Sy4J59wAZnLfp0Tw%2Fw02IMlVuOZWUnKchkzQWtY%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-2ffa76c8fe5848d9a5/Collect%20Devirtualization.EqualityComparer.ValueTupleCompareNoOpt/1/artifacts/BenchmarkDotNet.Artifacts/Devirtualization.EqualityComparer.ValueTupleCompareNoOpt-20230720-125644.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A04Z&sr=c&sp=rl&sig=GI9IN7SrwkmUpCUCz24JnCe%2Fb%2FGRhVXvTeMe8ApCVTI%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.526254762205227 < 8.959650675044514. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/4/2023 1:18:39 PM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 20.250307326640808 (T) = (0 -7.54997352252696) / Math.Sqrt((0.23579619703849655 / (299)) + (0.024247495467661386 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.14219407508841908 = (8.801493791623297 - 7.54997352252696) / 8.801493791623297 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Devirtualization_EqualityComparer_ValueTupleCompareNoOpt_baseline_97c9ce77-caf6-41fe-ad69-0727b22fdc88.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Devirtualization_EqualityComparer_ValueTupleCompareNoOpt_compare_97c9ce77-caf6-41fe-ad69-0727b22fdc88.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FDevirtualization_EqualityComparer_ValueTupleCompareNoOpt_baseline_97c9ce77-caf6-41fe-ad69-0727b22fdc88.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FDevirtualization_EqualityComparer_ValueTupleCompareNoOpt_compare_97c9ce77-caf6-41fe-ad69-0727b22fdc88.log) ### Devirtualization.EqualityComparer.ValueTupleCompareWrapped #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-f5e1d53d639442d588/Collect%20Devirtualization.EqualityComparer.ValueTupleCompareWrapped/1/artifacts/BenchmarkDotNet.Artifacts/Devirtualization.EqualityComparer.ValueTupleCompareWrapped-20230720-125821.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A07Z&sr=c&sp=rl&sig=8BGR3NQiO4yBUCboDcRPxjnSNpeDm7fBBUzK7hFy80Y%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-71cf0844bc004ba892/Collect%20Devirtualization.EqualityComparer.ValueTupleCompareWrapped/1/artifacts/BenchmarkDotNet.Artifacts/Devirtualization.EqualityComparer.ValueTupleCompareWrapped-20230720-125810.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A07Z&sr=c&sp=rl&sig=Id%2FuWVlDchUee5Iw7KDcqWCHeXJ6uUf7xuuK8886JA0%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.273407804367663 < 5.442168131906461. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/1/2023 6:36:20 PM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 26.865830803292255 (T) = (0 -4.380212448223086) / Math.Sqrt((0.22710740283232894 / (299)) + (0.035262809069290496 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.3059899791658919 = (6.311454181826728 - 4.380212448223086) / 6.311454181826728 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Devirtualization_EqualityComparer_ValueTupleCompareWrapped_baseline_1892e1cd-0720-4f74-9b04-ce8a17f5e43f.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Devirtualization_EqualityComparer_ValueTupleCompareWrapped_compare_1892e1cd-0720-4f74-9b04-ce8a17f5e43f.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FDevirtualization_EqualityComparer_ValueTupleCompareWrapped_baseline_1892e1cd-0720-4f74-9b04-ce8a17f5e43f.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FDevirtualization_EqualityComparer_ValueTupleCompareWrapped_compare_1892e1cd-0720-4f74-9b04-ce8a17f5e43f.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare d9f6b98de38081b71fa9b5a463f302288546435d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Fib

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[Test - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/Benchstone.BenchI.Fib.Test.html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 145.37 μs 103.86 μs 0.71 0.08 False 1361947.2789115645 985792.824074074 0.7238112952961703)

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Fib*'
### Payloads [Baseline]() [Compare]() ### Benchstone.BenchI.Fib.Test #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-2b975388ff8140dd98/Collect%20Benchstone.BenchI.Fib.Test/1/artifacts/BenchmarkDotNet.Artifacts/Benchstone.BenchI.Fib.Test-20230720-125555.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A04Z&sr=c&sp=rl&sig=jQFv3lYibrB9IUIE8vgMdhj3ONCemkAB89DzxVXUR7I%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-56de06e7c47246cf87/Collect%20Benchstone.BenchI.Fib.Test/1/artifacts/BenchmarkDotNet.Artifacts/Benchstone.BenchI.Fib.Test-20230720-125647.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A04Z&sr=c&sp=rl&sig=2A2jylskM0flDBJjHU1f%2BP%2Bby71JaQ5h7gB4%2BmQ5UGs%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 103.85542494481237 < 138.09473784722223. IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/1/2023 3:42:23 PM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 26.401742103191115 (T) = (0 -110088.22327440354) / Math.Sqrt((18834770.934165046 / (299)) + (14691038.36171529 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.2484137378735907 = (146474.50175970327 - 110088.22327440354) / 146474.50175970327 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Benchstone_BenchI_Fib_Test_baseline_d7769543-8d4e-42bf-9fce-facce197332e.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Benchstone_BenchI_Fib_Test_compare_d7769543-8d4e-42bf-9fce-facce197332e.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FBenchstone_BenchI_Fib_Test_baseline_d7769543-8d4e-42bf-9fce-facce197332e.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FBenchstone_BenchI_Fib_Test_compare_d7769543-8d4e-42bf-9fce-facce197332e.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare d9f6b98de38081b71fa9b5a463f302288546435d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[ImmutableList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.IterateFor(Int32).ImmutableList(Size%3a%20512).html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 14.28 μs 6.83 μs 0.48 0.16 False
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.IterateFor(Int32).ImmutableSortedSet(Size%3a%20512).html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 13.61 μs 6.50 μs 0.48 0.19 False 67816.17793736285 55219.36973648465 0.8142506908526604)

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateFor<Int32>.ImmutableList(Size: 512) #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-56dd81ec2f4544b3bf/Collect%20System.Collections.IterateFor_Int32_.ImmutableList_Size%3A%20512_/1/artifacts/BenchmarkDotNet.Artifacts/System.Collections.IterateFor_Int32_.ImmutableList%28Size_%20512%29-20230720-125704.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A05Z&sr=c&sp=rl&sig=BuMJ4vwEWkdNgKtOTpNXw%2BdTn238vN%2FLm3Aq6IYgoGw%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-87e54e95b57241f0ac/Collect%20System.Collections.IterateFor_Int32_.ImmutableList_Size%3A%20512_/1/artifacts/BenchmarkDotNet.Artifacts/System.Collections.IterateFor_Int32_.ImmutableList%28Size_%20512%29-20230720-125658.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A05Z&sr=c&sp=rl&sig=H5dN6JiYdL8Bc6fJ8fPR0CQuIR6N%2B%2Fds8RY0%2BBteSWY%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.834231225722918 < 12.867044369634995. IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 3:32:16 PM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 22.1810630776769 (T) = (0 -7139.676036700811) / Math.Sqrt((2713880.9013933064 / (299)) + (434192.256835829 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.4388170745800116 = (12722.546808346831 - 7139.676036700811) / 12722.546808346831 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/System_Collections_IterateFor_Int32__ImmutableList_Size__512__baseline_9f6785a8-8cd7-4a3b-b800-882bb96463de.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/System_Collections_IterateFor_Int32__ImmutableList_Size__512__compare_9f6785a8-8cd7-4a3b-b800-882bb96463de.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FSystem_Collections_IterateFor_Int32__ImmutableList_Size__512__baseline_9f6785a8-8cd7-4a3b-b800-882bb96463de.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FSystem_Collections_IterateFor_Int32__ImmutableList_Size__512__compare_9f6785a8-8cd7-4a3b-b800-882bb96463de.log) ### System.Collections.IterateFor<Int32>.ImmutableSortedSet(Size: 512) #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-ff087ee161894352b2/Collect%20System.Collections.IterateFor_Int32_.ImmutableSortedSet_Size%3A%20512_/1/artifacts/BenchmarkDotNet.Artifacts/System.Collections.IterateFor_Int32_.ImmutableSortedSet%28Size_%20512%29-20230720-125657.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A06Z&sr=c&sp=rl&sig=Yz9ri694jqyFUSbYY3BuAIB9NvNzDeszIelJoX7iuOQ%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-66271476a44b484d8f/Collect%20System.Collections.IterateFor_Int32_.ImmutableSortedSet_Size%3A%20512_/1/artifacts/BenchmarkDotNet.Artifacts/System.Collections.IterateFor_Int32_.ImmutableSortedSet%28Size_%20512%29-20230720-125752.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A05Z&sr=c&sp=rl&sig=2wVVBfwZ9zy1%2F3PMUKd6XEOv20SVxcbnpLkwn%2FLD42I%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.4982949320716665 < 13.108996364598319. IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/19/2023 10:30:38 AM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 29.50203636719508 (T) = (0 -6750.296658573149) / Math.Sqrt((3747765.1468149526 / (299)) + (220560.19704725235 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.4784078450514978 = (12941.714315545292 - 6750.296658573149) / 12941.714315545292 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/System_Collections_IterateFor_Int32__ImmutableSortedSet_Size__512__baseline_fd7a912f-b4ed-4edb-a553-4ef923cc68b3.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/System_Collections_IterateFor_Int32__ImmutableSortedSet_Size__512__compare_fd7a912f-b4ed-4edb-a553-4ef923cc68b3.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FSystem_Collections_IterateFor_Int32__ImmutableSortedSet_Size__512__baseline_fd7a912f-b4ed-4edb-a553-4ef923cc68b3.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FSystem_Collections_IterateFor_Int32__ImmutableSortedSet_Size__512__compare_fd7a912f-b4ed-4edb-a553-4ef923cc68b3.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare d9f6b98de38081b71fa9b5a463f302288546435d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.TreeSort

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[Test - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/Benchstone.BenchI.TreeSort.Test.html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 541.23 μs 499.13 μs 0.92 0.00 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.TreeSort*'
### Payloads [Baseline]() [Compare]() ### Benchstone.BenchI.TreeSort.Test #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-5a6627088ebb4737b4/Collect%20Benchstone.BenchI.TreeSort.Test/1/artifacts/BenchmarkDotNet.Artifacts/Benchstone.BenchI.TreeSort.Test-20230720-125712.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A07Z&sr=c&sp=rl&sig=H9xrWaHHSNmohAfWrMWkvmYYUoDCQSz1EK7VQ%2BLG6wI%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-90e2d48ed71f48de91/Collect%20Benchstone.BenchI.TreeSort.Test/1/artifacts/BenchmarkDotNet.Artifacts/Benchstone.BenchI.TreeSort.Test-20230720-125625.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A07Z&sr=c&sp=rl&sig=JkFV6yUiM5KFdLi0tKCemN6x6acQCMipJpQrVTxcg2M%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 499.1339713541667 < 524.0854280326355. IsChangePoint: Marked as a change because one of 5/12/2023 4:02:48 AM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 33.617721495890784 (T) = (0 -498805.5068299371) / Math.Sqrt((179911452.64478976 / (299)) + (17077235.510679174 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.10030432929400732 = (554415.8131143652 - 498805.5068299371) / 554415.8131143652 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Benchstone_BenchI_TreeSort_Test_baseline_dc78e1df-6bf4-4d29-83fe-1f606ff98bc0.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Benchstone_BenchI_TreeSort_Test_compare_dc78e1df-6bf4-4d29-83fe-1f606ff98bc0.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FBenchstone_BenchI_TreeSort_Test_baseline_dc78e1df-6bf4-4d29-83fe-1f606ff98bc0.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FBenchstone_BenchI_TreeSort_Test_compare_dc78e1df-6bf4-4d29-83fe-1f606ff98bc0.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare d9f6b98de38081b71fa9b5a463f302288546435d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[ImmutableList - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.ContainsTrue(Int32).ImmutableList(Size%3a%20512).html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 677.51 μs 532.87 μs 0.79 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrue<Int32>.ImmutableList(Size: 512) #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-27ff51f678ac45ff85/Collect%20System.Collections.ContainsTrue_Int32_.ImmutableList_Size%3A%20512_/1/artifacts/BenchmarkDotNet.Artifacts/System.Collections.ContainsTrue_Int32_.ImmutableList%28Size_%20512%29-20230720-125951.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A09Z&sr=c&sp=rl&sig=3hhq1kAar03du39iOD2vhqZ1hRdjvFLHydXs0SCBfyM%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-b6e1753a822c4e59b7/Collect%20System.Collections.ContainsTrue_Int32_.ImmutableList_Size%3A%20512_/1/artifacts/BenchmarkDotNet.Artifacts/System.Collections.ContainsTrue_Int32_.ImmutableList%28Size_%20512%29-20230720-125945.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A09Z&sr=c&sp=rl&sig=gXF0xlZdSFNUPEzUDtN5ecJH8XHmxVqBjQteMk5szMs%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 532.8685138888889 < 643.6256184895833. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 52.6338907624299 (T) = (0 -533575.0571295024) / Math.Sqrt((3339753621.088809 / (299)) + (2717348.603801331 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.25074236681177486 = (712138.2999583804 - 533575.0571295024) / 712138.2999583804 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/System_Collections_ContainsTrue_Int32__ImmutableList_Size__512__baseline_124be201-596e-4336-afdb-140d7d59f777.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/System_Collections_ContainsTrue_Int32__ImmutableList_Size__512__compare_124be201-596e-4336-afdb-140d7d59f777.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FSystem_Collections_ContainsTrue_Int32__ImmutableList_Size__512__baseline_124be201-596e-4336-afdb-140d7d59f777.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FSystem_Collections_ContainsTrue_Int32__ImmutableList_Size__512__compare_124be201-596e-4336-afdb-140d7d59f777.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare d9f6b98de38081b71fa9b5a463f302288546435d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in BenchmarksGame.BinaryTrees_2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[RunBench - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/BenchmarksGame.BinaryTrees_2.RunBench.html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 117.53 ms 87.56 ms 0.75 0.05 False 823533333.3333334 666588235.2941177 0.8094247170251571)

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.BinaryTrees_2*'
### Payloads [Baseline]() [Compare]() ### BenchmarksGame.BinaryTrees_2.RunBench #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-ccca1f41ff7642c382/Collect%20BenchmarksGame.BinaryTrees_2.RunBench/1/artifacts/BenchmarkDotNet.Artifacts/BenchmarksGame.BinaryTrees_2.RunBench-20230720-130027.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A11Z&sr=c&sp=rl&sig=ayr97QeV9KbTDeeK9RLRmpQFQp2%2FvTFD5Eb6AxXG7OM%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-2f9a5c9375b844f1b4/Collect%20BenchmarksGame.BinaryTrees_2.RunBench/1/artifacts/BenchmarkDotNet.Artifacts/BenchmarksGame.BinaryTrees_2.RunBench-20230720-130023.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A11Z&sr=c&sp=rl&sig=llJ0h0r2%2FqBA6oHytmCvSMYstvd%2BeJ4UX%2FwKmmN61pk%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 87.56414615384615 < 111.6755418095238. IsChangePoint: Marked as a change because one of 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 72.77460549904681 (T) = (0 -89196562.43847299) / Math.Sqrt((14586336135848.738 / (299)) + (854849034888.9264 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.24350090269197058 = (117907030.89517918 - 89196562.43847299) / 117907030.89517918 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/BenchmarksGame_BinaryTrees_2_RunBench_baseline_b6b698cf-d31a-4a49-b908-d050d3b5b808.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/BenchmarksGame_BinaryTrees_2_RunBench_compare_b6b698cf-d31a-4a49-b908-d050d3b5b808.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FBenchmarksGame_BinaryTrees_2_RunBench_baseline_b6b698cf-d31a-4a49-b908-d050d3b5b808.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FBenchmarksGame_BinaryTrees_2_RunBench_compare_b6b698cf-d31a-4a49-b908-d050d3b5b808.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare d9f6b98de38081b71fa9b5a463f302288546435d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.ContainsFalse(Int32).ImmutableSortedSet(Size%3a%20512).html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 34.94 μs 22.62 μs 0.65 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsFalse<Int32>.ImmutableSortedSet(Size: 512) #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-8edd0e6d5f55487c95/Collect%20System.Collections.ContainsFalse_Int32_.ImmutableSortedSet_Size%3A%20512_/1/artifacts/BenchmarkDotNet.Artifacts/System.Collections.ContainsFalse_Int32_.ImmutableSortedSet%28Size_%20512%29-20230720-130042.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A12Z&sr=c&sp=rl&sig=U0b2D8oRtphqShjZUjQe7wSF%2B39gkjFDDQzJroX6OEA%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-23baa405186f4226ad/Collect%20System.Collections.ContainsFalse_Int32_.ImmutableSortedSet_Size%3A%20512_/1/artifacts/BenchmarkDotNet.Artifacts/System.Collections.ContainsFalse_Int32_.ImmutableSortedSet%28Size_%20512%29-20230720-130201.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A12Z&sr=c&sp=rl&sig=%2BiH3omvcKmhmL1FsZapkk1uMpg2JEofHANXb8qM%2BEmc%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.61515293834297 < 32.98689070436027. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 50.513552759583945 (T) = (0 -22630.751995236613) / Math.Sqrt((28496346.500366475 / (299)) + (6481.049496898382 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.40898337428716086 = (38291.22737104244 - 22630.751995236613) / 38291.22737104244 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/System_Collections_ContainsFalse_Int32__ImmutableSortedSet_Size__512__baseline_7bbc951b-7fca-4c0b-af87-c10cc8a0aea5.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/System_Collections_ContainsFalse_Int32__ImmutableSortedSet_Size__512__compare_7bbc951b-7fca-4c0b-af87-c10cc8a0aea5.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FSystem_Collections_ContainsFalse_Int32__ImmutableSortedSet_Size__512__baseline_7bbc951b-7fca-4c0b-af87-c10cc8a0aea5.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FSystem_Collections_ContainsFalse_Int32__ImmutableSortedSet_Size__512__compare_7bbc951b-7fca-4c0b-af87-c10cc8a0aea5.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline d59af2cf097acb100ad6a9cba652be34be6f4a2e
Compare d9f6b98de38081b71fa9b5a463f302288546435d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchI.Ackermann

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
[Test - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/Benchstone.BenchI.Ackermann.Test.html>)
📝 - Benchmark Source
ADX - Test Multi COnfig Graph		 4.17 μs 2.71 μs 0.65 0.27 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Ackermann*'
### Payloads [Baseline]() [Compare]() ### Benchstone.BenchI.Ackermann.Test #### ETL Files [Baseline](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-3b3344a49ceb4ab59b/Collect%20Benchstone.BenchI.Ackermann.Test/1/artifacts/BenchmarkDotNet.Artifacts/Benchstone.BenchI.Ackermann.Test-20230720-130129.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A13Z&sr=c&sp=rl&sig=mISb6CMNe4CNESfMfeEQWig17r7oPWnE3h2lLGxvXWE%3D) [Compare](https://helixri107v0xdeko0k025g8.blob.core.windows.net/results-8beb1ca728ef43a2b9/Collect%20Benchstone.BenchI.Ackermann.Test/1/artifacts/BenchmarkDotNet.Artifacts/Benchstone.BenchI.Ackermann.Test-20230720-130116.etl?sv=2021-08-06&se=2023-10-18T19%3A54%3A13Z&sr=c&sp=rl&sig=bmkpSiw21yJvJh4dmqmH6x4ZM3jjpwXcF8V3GhDobvI%3D) #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.706582943925234 < 3.9745923370866882. IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 7/12/2023 4:12:15 PM, 7/17/2023 10:23:26 PM falls between 7/8/2023 10:33:35 PM and 7/17/2023 10:23:26 PM. IsImprovementStdDev: Marked as improvement because 74.5411058456886 (T) = (0 -2666.0840896238597) / Math.Sqrt((40979.75815125292 / (299)) + (3713.911485614174 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.40673664489609596 = (4493.930168933023 - 2666.0840896238597) / 4493.930168933023 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms [Baseline](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Benchstone_BenchI_Ackermann_Test_baseline_79ad11b9-bc22-4121-8ced-85ffa62c34be.log) [Compare](https://pvscmdupload.blob.core.windows.net/autofilereport/jitdasms/07_18_2023/Benchstone_BenchI_Ackermann_Test_compare_79ad11b9-bc22-4121-8ced-85ffa62c34be.log) [Diff](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FBenchstone_BenchI_Ackermann_Test_baseline_79ad11b9-bc22-4121-8ced-85ffa62c34be.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2Fjitdasms%2F07_18_2023%2FBenchstone_BenchI_Ackermann_Test_compare_79ad11b9-bc22-4121-8ced-85ffa62c34be.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)