search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x86: 21 Improvements on 4/17/2023 1:30:24 AM #16472

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FormatterInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%209223372036854775807).html>) 67.30 ns 53.42 ns 0.79 0.01 True Trace Trace
[FormatterDouble - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value%3a%2012345).html>) 213.50 ns 201.76 ns 0.95 0.00 False 1741.2970307240835 1609.2553931802368 0.924170526214623) Trace Trace
[FormatterUInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value%3a%204294967295).html>) 74.76 ns 15.65 ns 0.21 0.06 False 391.14101413129913 130.1323519578881 0.3326993264741216) Trace Trace
[FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%2018446744073709551615).html>) 68.63 ns 53.68 ns 0.78 0.01 False Trace Trace
[FormatterInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%202147483647).html>) 19.65 ns 16.93 ns 0.86 0.07 False 227.56734352115478 136.9665425270688 0.6018725727856304) Trace Trace

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 53.42378135369594 < 63.92617324343382. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 48.84388934765177 (T) = (0 -53.964136007881436) / Math.Sqrt((0.7696639632785475 / (299)) + (0.20914530710280432 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.16017690629239914 = (64.25655166214088 - 53.964136007881436) / 64.25655166214088 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Buffers_Text_Tests_Utf8FormatterTests_FormatterInt64_value__9223372036854775807__baseline_dea9a250-2c55-418f-89f8-89143fd73c20.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Buffers_Text_Tests_Utf8FormatterTests_FormatterInt64_value__9223372036854775807__compare_dea9a250-2c55-418f-89f8-89143fd73c20.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Buffers_Text_Tests_Utf8FormatterTests_FormatterInt64_value__9223372036854775807__baseline_dea9a250-2c55-418f-89f8-89143fd73c20.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Buffers_Text_Tests_Utf8FormatterTests_FormatterInt64_value__9223372036854775807__compare_dea9a250-2c55-418f-89f8-89143fd73c20.log) #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 12345) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 201.7646491778423 < 203.10606142457144. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 56.35702640544221 (T) = (0 -202.23658226273565) / Math.Sqrt((2.1235491175834955 / (299)) + (0.22016876802128837 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.05928046118748982 = (214.9807396559692 - 202.23658226273565) / 214.9807396559692 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Buffers_Text_Tests_Utf8FormatterTests_FormatterDouble_value__12345__baseline_dcfcfcea-d6b3-4750-b1da-0b0cbe4a3384.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Buffers_Text_Tests_Utf8FormatterTests_FormatterDouble_value__12345__compare_dcfcfcea-d6b3-4750-b1da-0b0cbe4a3384.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Buffers_Text_Tests_Utf8FormatterTests_FormatterDouble_value__12345__baseline_dcfcfcea-d6b3-4750-b1da-0b0cbe4a3384.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Buffers_Text_Tests_Utf8FormatterTests_FormatterDouble_value__12345__compare_dcfcfcea-d6b3-4750-b1da-0b0cbe4a3384.log) #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 4294967295) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.653677634460239 < 71.02397633239009. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 6.152492177280173 (T) = (0 -15.445705538988388) / Math.Sqrt((180.0322332337118 / (299)) + (0.015696841616112733 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.23657900573857554 = (20.2322252794887 - 15.445705538988388) / 20.2322252794887 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Buffers_Text_Tests_Utf8FormatterTests_FormatterUInt32_value__4294967295__baseline_f2ba5a6c-73a4-4ace-8719-391b6f96e655.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Buffers_Text_Tests_Utf8FormatterTests_FormatterUInt32_value__4294967295__compare_f2ba5a6c-73a4-4ace-8719-391b6f96e655.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Buffers_Text_Tests_Utf8FormatterTests_FormatterUInt32_value__4294967295__baseline_f2ba5a6c-73a4-4ace-8719-391b6f96e655.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Buffers_Text_Tests_Utf8FormatterTests_FormatterUInt32_value__4294967295__compare_f2ba5a6c-73a4-4ace-8719-391b6f96e655.log) #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 53.677622061482815 < 64.92811670774262. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 87.91846817040938 (T) = (0 -53.667863816237784) / Math.Sqrt((1.1439285194003936 / (299)) + (0.06142674672783015 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.17214157876323063 = (64.827345400511 - 53.667863816237784) / 64.827345400511 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Buffers_Text_Tests_Utf8FormatterTests_FormatterUInt64_value__18446744073709551615__baseline_687275f0-abca-4501-8c64-24c0129ace43.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Buffers_Text_Tests_Utf8FormatterTests_FormatterUInt64_value__18446744073709551615__compare_687275f0-abca-4501-8c64-24c0129ace43.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Buffers_Text_Tests_Utf8FormatterTests_FormatterUInt64_value__18446744073709551615__baseline_687275f0-abca-4501-8c64-24c0129ace43.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Buffers_Text_Tests_Utf8FormatterTests_FormatterUInt64_value__18446744073709551615__compare_687275f0-abca-4501-8c64-24c0129ace43.log) #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 16.930863885868927 < 18.656853272505543. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 47.1651538305922 (T) = (0 -16.93654796680112) / Math.Sqrt((0.6839990316184374 / (299)) + (0.03690808345136845 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.21497034995918896 = (21.574405458342426 - 16.93654796680112) / 21.574405458342426 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Buffers_Text_Tests_Utf8FormatterTests_FormatterInt32_value__2147483647__baseline_240c51f4-66b6-4fcf-bc92-135e622763d1.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Buffers_Text_Tests_Utf8FormatterTests_FormatterInt32_value__2147483647__compare_240c51f4-66b6-4fcf-bc92-135e622763d1.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Buffers_Text_Tests_Utf8FormatterTests_FormatterInt32_value__2147483647__baseline_240c51f4-66b6-4fcf-bc92-135e622763d1.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Buffers_Text_Tests_Utf8FormatterTests_FormatterInt32_value__2147483647__compare_240c51f4-66b6-4fcf-bc92-135e622763d1.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tasks.ValueTaskPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Copy_PassAsArgumentAndReturn_FromValueTaskSource - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Threading.Tasks.ValueTaskPerfTest.Copy_PassAsArgumentAndReturn_FromValueTaskSource.html>) 15.65 ns 14.57 ns 0.93 0.00 False Trace Trace

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Tasks.ValueTaskPerfTest.Copy_PassAsArgumentAndReturn_FromValueTaskSource ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.567912495514229 < 14.829182677489346. IsChangePoint: Marked as a change because one of 2/12/2023 9:02:44 AM, 3/25/2023 10:49:06 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 8.499216953023266 (T) = (0 -14.6877654093466) / Math.Sqrt((0.0342067800850525 / (299)) + (0.062284257624813086 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.06092727318380064 = (15.640711299478909 - 14.6877654093466) / 15.640711299478909 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Threading_Tasks_ValueTaskPerfTest_Copy_PassAsArgumentAndReturn_FromValueTaskSource_baseline_f8034a17-4d41-4c71-b687-45728a1f14a3.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Threading_Tasks_ValueTaskPerfTest_Copy_PassAsArgumentAndReturn_FromValueTaskSource_compare_f8034a17-4d41-4c71-b687-45728a1f14a3.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Threading_Tasks_ValueTaskPerfTest_Copy_PassAsArgumentAndReturn_FromValueTaskSource_baseline_f8034a17-4d41-4c71-b687-45728a1f14a3.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Threading_Tasks_ValueTaskPerfTest_Copy_PassAsArgumentAndReturn_FromValueTaskSource_compare_f8034a17-4d41-4c71-b687-45728a1f14a3.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_DateTime.ToString(format%3a%20null).html>) 358.04 ns 299.26 ns 0.84 0.01 False 3035.986833636748 2553.199735015605 0.8409785268920873) Trace Trace
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_DateTime.ToString(format%3a%20%22G%22).html>) 354.71 ns 298.45 ns 0.84 0.02 False Trace Trace
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_DateTime.ToString(format%3a%20%22s%22).html>) 326.46 ns 284.46 ns 0.87 0.00 False 2733.991098756072 2363.3924941390146 0.8644477647400993) Trace Trace

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_DateTime.ToString(format: null) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 299.2559491435464 < 337.1356616959433. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 17.165729022638722 (T) = (0 -302.5861785091913) / Math.Sqrt((61.37137984071002 / (299)) + (15.67023938558174 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.09392971192087256 = (333.95442107551634 - 302.5861785091913) / 333.95442107551634 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_DateTime_ToString_format__null__baseline_0194723a-4bd6-406f-9635-8fd9c85bf590.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_DateTime_ToString_format__null__compare_0194723a-4bd6-406f-9635-8fd9c85bf590.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_DateTime_ToString_format__null__baseline_0194723a-4bd6-406f-9635-8fd9c85bf590.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_DateTime_ToString_format__null__compare_0194723a-4bd6-406f-9635-8fd9c85bf590.log) #### System.Tests.Perf_DateTime.ToString(format: "G") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 298.44591297542166 < 337.2571109205692. IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 4.246828991649039 (T) = (0 -307.3647163074625) / Math.Sqrt((60.77614668269057 / (299)) + (177.8407525109642 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07633006478080838 = (332.7646647224945 - 307.3647163074625) / 332.7646647224945 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_DateTime_ToString_format___G___baseline_38f64764-c09b-4dfe-b6b4-22db8dc4fcb2.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_DateTime_ToString_format___G___compare_38f64764-c09b-4dfe-b6b4-22db8dc4fcb2.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_DateTime_ToString_format___G___baseline_38f64764-c09b-4dfe-b6b4-22db8dc4fcb2.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_DateTime_ToString_format___G___compare_38f64764-c09b-4dfe-b6b4-22db8dc4fcb2.log) #### System.Tests.Perf_DateTime.ToString(format: "s") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 284.4648720816945 < 309.53525108768366. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 34.34537510283639 (T) = (0 -283.2937980092015) / Math.Sqrt((17.691670672360694 / (299)) + (6.8918564629319805 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12691113552249772 = (324.47303995652027 - 283.2937980092015) / 324.47303995652027 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_DateTime_ToString_format___s___baseline_eacea633-0f85-465a-9958-792ccc5b9ccc.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_DateTime_ToString_format___s___compare_eacea633-0f85-465a-9958-792ccc5b9ccc.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_DateTime_ToString_format___s___baseline_eacea633-0f85-465a-9958-792ccc5b9ccc.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_DateTime_ToString_format___s___compare_eacea633-0f85-465a-9958-792ccc5b9ccc.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.Perf_RandomAccess

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadAsync - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.IO.Tests.Perf_RandomAccess.ReadAsync(fileSize%3a%201048576%2c%20bufferSize%3a%204096%2c%20options%3a%20Asynchronous).html>) 4.13 ms 2.53 ms 0.61 0.35 False Trace Trace

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_RandomAccess*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_RandomAccess.ReadAsync(fileSize: 1048576, bufferSize: 4096, options: Asynchronous) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.5281751736111104 < 3.0986057575827206. IsChangePoint: Marked as a change because one of 3/3/2023 9:00:24 PM, 3/4/2023 7:07:20 AM, 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 4.29941924195415 (T) = (0 -2519327.6945945593) / Math.Sqrt((1005677860276.9319 / (299)) + (3176701367.090229 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.09740545897583416 = (2791206.438868887 - 2519327.6945945593) / 2791206.438868887 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_IO_Tests_Perf_RandomAccess_ReadAsync_fileSize__1048576__bufferSize__4096__options__Asynchrono_baseline_220cff15-7894-4b6c-9213-32e2b5b1e03c.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_IO_Tests_Perf_RandomAccess_ReadAsync_fileSize__1048576__bufferSize__4096__options__Asynchrono_compare_220cff15-7894-4b6c-9213-32e2b5b1e03c.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_IO_Tests_Perf_RandomAccess_ReadAsync_fileSize__1048576__bufferSize__4096__options__Asynchrono_baseline_220cff15-7894-4b6c-9213-32e2b5b1e03c.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_IO_Tests_Perf_RandomAccess_ReadAsync_fileSize__1048576__bufferSize__4096__options__Asynchrono_compare_220cff15-7894-4b6c-9213-32e2b5b1e03c.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_UInt64.TryFormat(value%3a%200).html>) 12.09 ns 7.77 ns 0.64 0.51 False 77.21250804096262 79.33801119943197 1.0275279642171673) Trace Trace

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_UInt64.TryFormat(value: 0) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.771741995683265 < 11.49588192032264. IsChangePoint: Marked as a change because one of 2/7/2023 8:22:47 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 55.480419174743076 (T) = (0 -7.848302279047765) / Math.Sqrt((1.2660152046810291 / (299)) + (0.0021286810043205403 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.3254915613440949 = (11.635587976759934 - 7.848302279047765) / 11.635587976759934 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_UInt64_TryFormat_value__0__baseline_69f9f88b-d1be-404f-be19-e19719726bf2.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_UInt64_TryFormat_value__0__compare_69f9f88b-d1be-404f-be19-e19719726bf2.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_UInt64_TryFormat_value__0__baseline_69f9f88b-d1be-404f-be19-e19719726bf2.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_UInt64_TryFormat_value__0__compare_69f9f88b-d1be-404f-be19-e19719726bf2.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForEachNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.IterateForEachNonGeneric(String).Stack(Size%3a%20512).html>) 3.68 μs 3.41 μs 0.93 0.02 False Trace Trace

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEachNonGeneric<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.4103832714566495 < 3.502078628882798. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 5:47:38 PM, 3/6/2023 11:14:52 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 55.09227946061861 (T) = (0 -3402.46967235863) / Math.Sqrt((8357.509480960323 / (299)) + (67.19423675047564 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.09434305374584831 = (3756.907829649446 - 3402.46967235863) / 3756.907829649446 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_IterateForEachNonGeneric_String__Stack_Size__512__baseline_31eb8fd8-7535-4996-8d06-ecad38e8054d.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_IterateForEachNonGeneric_String__Stack_Size__512__compare_31eb8fd8-7535-4996-8d06-ecad38e8054d.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_IterateForEachNonGeneric_String__Stack_Size__512__baseline_31eb8fd8-7535-4996-8d06-ecad38e8054d.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_IterateForEachNonGeneric_String__Stack_Size__512__compare_31eb8fd8-7535-4996-8d06-ecad38e8054d.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20%22s%22).html>) 340.56 ns 294.17 ns 0.86 0.00 False Trace Trace
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20%22G%22).html>) 376.90 ns 333.50 ns 0.88 0.02 False 3135.5851450638183 2650.044954555221 0.8451516485613676) Trace Trace

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_DateTimeOffset.ToString(format: "s") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 294.16726560963474 < 320.8827575884048. IsChangePoint: Marked as a change because one of 3/15/2023 5:33:41 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 74.37157380937433 (T) = (0 -295.6587364899473) / Math.Sqrt((14.941849735046622 / (299)) + (1.2956371704622356 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1226924275154955 = (337.0069354954409 - 295.6587364899473) / 337.0069354954409 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_DateTimeOffset_ToString_format___s___baseline_5b1c5b65-18c7-400e-8446-876fdf1033a1.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_DateTimeOffset_ToString_format___s___compare_5b1c5b65-18c7-400e-8446-876fdf1033a1.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_DateTimeOffset_ToString_format___s___baseline_5b1c5b65-18c7-400e-8446-876fdf1033a1.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_DateTimeOffset_ToString_format___s___compare_5b1c5b65-18c7-400e-8446-876fdf1033a1.log) #### System.Tests.Perf_DateTimeOffset.ToString(format: "G") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 333.5012825185874 < 350.96258446394177. IsChangePoint: Marked as a change because one of 3/15/2023 5:33:41 AM, 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 6.429137148662151 (T) = (0 -315.75201609828747) / Math.Sqrt((66.49960208150429 / (299)) + (99.55124854483383 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08371224372721502 = (344.59918724951945 - 315.75201609828747) / 344.59918724951945 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_DateTimeOffset_ToString_format___G___baseline_81e8a665-6502-4876-a0f6-9688abf860b6.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Tests_Perf_DateTimeOffset_ToString_format___G___compare_81e8a665-6502-4876-a0f6-9688abf860b6.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_DateTimeOffset_ToString_format___G___baseline_81e8a665-6502-4876-a0f6-9688abf860b6.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Tests_Perf_DateTimeOffset_ToString_format___G___compare_81e8a665-6502-4876-a0f6-9688abf860b6.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20da).html>) 298.18 ns 245.91 ns 0.82 0.00 True 2475.6574002417465 2097.7578564274454 0.8473538609270412) Trace Trace
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20fr).html>) 328.90 ns 260.97 ns 0.79 0.01 False 2765.7774056547837 2209.5456259888438 0.798887727360599) Trace Trace
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20ja).html>) 330.31 ns 254.65 ns 0.77 0.01 False Trace Trace
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20).html>) 331.26 ns 261.25 ns 0.79 0.01 False 2766.912317962784 2208.344493271748 0.798125939494065) Trace Trace
[ToStringHebrewIsrael - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael.html>) 505.88 ns 461.97 ns 0.91 0.00 False Trace Trace

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: da) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 245.90765984813143 < 282.8257159870408. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 24.05658494803355 (T) = (0 -247.1630313258166) / Math.Sqrt((8.587243360454387 / (299)) + (17.30791807731436 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15385956723733127 = (292.10639482009316 - 247.1630313258166) / 292.10639482009316 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__da__baseline_c89f1d0c-0660-4dad-b729-d01a8f9004f3.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__da__compare_c89f1d0c-0660-4dad-b729-d01a8f9004f3.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__da__baseline_c89f1d0c-0660-4dad-b729-d01a8f9004f3.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__da__compare_c89f1d0c-0660-4dad-b729-d01a8f9004f3.log) #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: fr) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 260.9675111486113 < 312.89474532526725. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 84.61039922140772 (T) = (0 -260.3664058454989) / Math.Sqrt((56.3656416123238 / (299)) + (0.16131419435015365 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13246533520185247 = (300.1221927034804 - 260.3664058454989) / 300.1221927034804 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__fr__baseline_6625db9a-af78-4312-9953-36da11cb08ff.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__fr__compare_6625db9a-af78-4312-9953-36da11cb08ff.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__fr__baseline_6625db9a-af78-4312-9953-36da11cb08ff.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__fr__compare_6625db9a-af78-4312-9953-36da11cb08ff.log) #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ja) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 254.64557567212847 < 313.8741145547457. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 74.05725017694157 (T) = (0 -255.3331649849781) / Math.Sqrt((64.26593013441563 / (299)) + (0.5851075831025095 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1431825392366024 = (298.0018226489972 - 255.3331649849781) / 298.0018226489972 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__ja__baseline_36e7cc32-b52b-4579-845b-d27eb70c662b.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__ja__compare_36e7cc32-b52b-4579-845b-d27eb70c662b.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__ja__baseline_36e7cc32-b52b-4579-845b-d27eb70c662b.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring__ja__compare_36e7cc32-b52b-4579-845b-d27eb70c662b.log) #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 261.2473759944058 < 314.3285962309632. IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 73.01763486698648 (T) = (0 -260.4087860037622) / Math.Sqrt((56.663741890063065 / (299)) + (0.5957251676812058 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13478281968140915 = (300.9750521919529 - 260.4087860037622) / 300.9750521919529 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring____baseline_79cc623a-111e-4246-83a4-f121550235d9.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring____compare_79cc623a-111e-4246-83a4-f121550235d9.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring____baseline_79cc623a-111e-4246-83a4-f121550235d9.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Globalization_Tests_Perf_DateTimeCultureInfo_ToString_culturestring____compare_79cc623a-111e-4246-83a4-f121550235d9.log) #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 461.96883223066504 < 480.4113868170534. IsChangePoint: Marked as a change because one of 2/17/2023 5:12:50 PM, 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 53.924548930518995 (T) = (0 -463.11495252864387) / Math.Sqrt((19.49780236802217 / (299)) + (0.7739556329128634 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.0517864756955845 = (488.4078750810613 - 463.11495252864387) / 488.4078750810613 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Globalization_Tests_Perf_DateTimeCultureInfo_ToStringHebrewIsrael_baseline_b673d2cc-2538-47bb-819b-e3b7dfcda9a3.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Globalization_Tests_Perf_DateTimeCultureInfo_ToStringHebrewIsrael_compare_b673d2cc-2538-47bb-819b-e3b7dfcda9a3.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Globalization_Tests_Perf_DateTimeCultureInfo_ToStringHebrewIsrael_baseline_b673d2cc-2538-47bb-819b-e3b7dfcda9a3.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Globalization_Tests_Perf_DateTimeCultureInfo_ToStringHebrewIsrael_compare_b673d2cc-2538-47bb-819b-e3b7dfcda9a3.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.Perf_SortedSet

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Min - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Collections.Tests.Perf_SortedSet.Min.html>) 19.77 ns 12.52 ns 0.63 0.24 False Trace Trace

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_SortedSet*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_SortedSet.Min ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.515463726958581 < 18.77828074086596. IsChangePoint: Marked as a change because one of 2/14/2023 9:29:25 PM, 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 8.363020446989616 (T) = (0 -12.542890084945991) / Math.Sqrt((9.121876702492537 / (299)) + (0.014003973973579682 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10848580826698603 = (14.069198450519194 - 12.542890084945991) / 14.069198450519194 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_Tests_Perf_SortedSet_Min_baseline_640a3f2d-d336-41d2-a0d2-cfc4b0f504fe.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Collections_Tests_Perf_SortedSet_Min_compare_640a3f2d-d336-41d2-a0d2-cfc4b0f504fe.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_Tests_Perf_SortedSet_Min_baseline_640a3f2d-d336-41d2-a0d2-cfc4b0f504fe.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Collections_Tests_Perf_SortedSet_Min_compare_640a3f2d-d336-41d2-a0d2-cfc4b0f504fe.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Numerics.Tests.Perf_VectorOf(Int64).DivideBenchmark.html>) 26.26 ns 20.76 ns 0.79 0.06 False Trace Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int64>.DivideBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 20.756303272362945 < 24.934950465885926. IsChangePoint: Marked as a change because one of 2/4/2023 4:58:40 AM, 2/6/2023 9:02:07 PM, 2/28/2023 10:21:25 PM, 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsImprovementStdDev: Marked as improvement because 25.209974468098846 (T) = (0 -21.178235327424694) / Math.Sqrt((2.8472663580020923 / (299)) + (0.05659535615771105 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1466483702428887 = (24.817712404735943 - 21.178235327424694) / 24.817712404735943 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Baseline JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Numerics_Tests_Perf_VectorOf_Int64__DivideBenchmark_baseline_2a742188-ad57-42d8-90bc-be81afd4f4f5.log) ### Compare JIT Disasm [Complete output](https://pvscmdupload.z22.web.core.windows.net/autofilereport/04_18_2023/System_Numerics_Tests_Perf_VectorOf_Int64__DivideBenchmark_compare_2a742188-ad57-42d8-90bc-be81afd4f4f5.log) ### JIT Disasm Diff [Link](https://perfsupport.azurewebsites.net/diff?old=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Numerics_Tests_Perf_VectorOf_Int64__DivideBenchmark_baseline_2a742188-ad57-42d8-90bc-be81afd4f4f5.log&new=https%3A%2F%2Fpvscmdupload.blob.core.windows.net%2Fautofilereport%2F04_18_2023%2FSystem_Numerics_Tests_Perf_VectorOf_Int64__DivideBenchmark_compare_2a742188-ad57-42d8-90bc-be81afd4f4f5.log) ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)