search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 64 Regressions on 4/17/2023 1:30:24 AM #16618

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteDeepUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20True).html>) 4.69 ms 5.38 ms 1.15 0.08 False
[WriteDeepUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True).html>) 4.75 ms 5.32 ms 1.12 0.08 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Deep*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.3844755319148945 > 4.9217904767779395. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -24.85725056481444 (T) = (0 -5356727.258358663) / Math.Sqrt((7708137457.475494 / (31)) + (2707043114.82794 / (5))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (5) - 2, .025) and -0.1499959027510171 = (4658040.298703947 - 5356727.258358663) / 4658040.298703947 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.324182411347518 > 4.990909875. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -16.071443360647667 (T) = (0 -5331110.6292369375) / Math.Sqrt((12778812680.57258 / (32)) + (4079505441.9471164 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.11743280988332694 = (4770855.645265658 - 5331110.6292369375) / 4770855.645265658 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FormatterUInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value%3a%200).html>) 1.96 ns 9.38 ns 4.78 0.07 True
[FormatterInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%204).html>) 1.99 ns 10.23 ns 5.14 0.09 True
[FormatterDecimal - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value%3a%20123456.789).html>) 30.94 ns 58.07 ns 1.88 0.02 True
[FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%200).html>) 1.96 ns 9.34 ns 4.77 0.06 True
[FormatterInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%2012345).html>) 9.65 ns 12.96 ns 1.34 0.01 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 0) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.382178159445944 > 2.0635795485277297. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -48.259446837994886 (T) = (0 -9.381064288928268) / Math.Sqrt((0.8450349977499843 / (32)) + (9.635421538974886E-06 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -5.097715184253685 = (1.5384556355064394 - 9.381064288928268) / 1.5384556355064394 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 4) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.225280438150744 > 2.089237524170981. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -125.12077004669628 (T) = (0 -10.207057465579089) / Math.Sqrt((0.13391523207730746 / (31)) + (0.0007698752070740558 / (5))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (5) - 2, .025) and -4.552894356471647 = (1.838150847167338 - 10.207057465579089) / 1.838150847167338 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value: 123456.789) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 58.07144768946109 > 32.49587631497417. IsChangePoint: Marked as a change because one of 2/20/2023 10:31:48 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -356.1499816590029 (T) = (0 -57.98898210600173) / Math.Sqrt((0.13230543395514738 / (32)) + (0.007882200830765641 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.866136560912093 = (31.07435078473519 - 57.98898210600173) / 31.07435078473519 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 0) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.342818285905707 > 2.051306378298528. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -36.78379604606133 (T) = (0 -9.345453204884361) / Math.Sqrt((1.409826470193561 / (32)) + (7.963870764782744E-06 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -4.752901136543442 = (1.6244765872162126 - 9.345453204884361) / 1.6244765872162126 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 12345) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.961185974733244 > 10.175667439487187. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -7.800743545674917 (T) = (0 -12.968501573444218) / Math.Sqrt((2.05864983152177 / (32)) + (3.15892456035132E-05 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.18004562653768652 = (10.989830631798922 - 12.968501573444218) / 10.989830631798922 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%2010).html>) 777.11 ns 840.76 ns 1.08 0.04 False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>) 1.74 ms 2.45 ms 1.40 0.01 True
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%2010).html>) 745.09 ns 809.18 ns 1.09 0.06 False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>) 2.25 ms 3.02 ms 1.34 0.01 True
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>) 2.34 ms 3.17 ms 1.35 0.01 True
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>) 2.39 ms 3.13 ms 1.31 0.01 True
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%2010).html>) 870.85 ns 983.10 ns 1.13 0.06 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%2010).html>) 652.32 ns 736.74 ns 1.13 0.01 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%2010).html>) 812.22 ns 901.32 ns 1.11 0.07 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>) 1.75 ms 2.42 ms 1.38 0.00 True
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>) 1.73 ms 2.28 ms 1.32 0.15 True
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%2010).html>) 610.32 ns 688.79 ns 1.13 0.09 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%2010).html>) 787.36 ns 852.29 ns 1.08 0.06 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>) 2.25 ms 3.00 ms 1.34 0.01 True
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%2010).html>) 939.94 ns 999.62 ns 1.06 0.04 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 840.7597854632526 > 802.6857470862607. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -25.485231959586876 (T) = (0 -849.1378502846121) / Math.Sqrt((110.47636402788585 / (32)) + (35.662026469802385 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.10821171635284263 = (766.2234911927748 - 849.1378502846121) / 766.2234911927748 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4450395424836593 > 1.8377644160839162. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -121.33959118885532 (T) = (0 -2434839.5453844965) / Math.Sqrt((378614254.1082931 / (32)) + (102089154.50133184 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.39470953892817534 = (1745768.1885904744 - 2434839.5453844965) / 1745768.1885904744 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 809.1842581086814 > 776.2724759441825. IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 4/3/2023 7:13:05 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -15.10539594803062 (T) = (0 -801.189700138618) / Math.Sqrt((413.88517092965657 / (32)) + (40.23600957734827 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.09452262313307196 = (731.999214274084 - 801.189700138618) / 731.999214274084 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0172832128514058 > 2.349155983660934. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -81.07512756721212 (T) = (0 -3004221.119583131) / Math.Sqrt((2534446517.9096417 / (32)) + (98654058.80261916 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.3669158840136631 = (2197809.7955536684 - 3004221.119583131) / 2197809.7955536684 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.1748025641025643 > 2.4473948654117437. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -10.924498926823173 (T) = (0 -3312804.915020361) / Math.Sqrt((2031778904.860919 / (32)) + (42930033282.15954 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.442358501721536 = (2296797.163164596 - 3312804.915020361) / 2296797.163164596 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.1324678333333336 > 2.508483666666667. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -63.058650809374015 (T) = (0 -3149333.491984668) / Math.Sqrt((2733979557.7115355 / (32)) + (438904664.25964 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.35781984823638124 = (2319404.51899802 - 3149333.491984668) / 2319404.51899802 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 983.1028900821225 > 923.9769099641784. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -27.29707902840894 (T) = (0 -976.9301048863584) / Math.Sqrt((197.0904510247513 / (32)) + (25.509275746510014 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.10346630070252566 = (885.3284457027754 - 976.9301048863584) / 885.3284457027754 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 736.7405083579088 > 685.2713084812452. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -37.77402444588009 (T) = (0 -734.6852729964257) / Math.Sqrt((59.82824893681949 / (32)) + (9.161894450527722 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.10978686806129892 = (662.0057365427805 - 734.6852729964257) / 662.0057365427805 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 901.3243829203482 > 855.715431469665. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -18.399106993723755 (T) = (0 -904.926228313356) / Math.Sqrt((360.84914279814353 / (32)) + (60.20582296139446 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.10886967701888998 = (816.0798758120791 - 904.926228313356) / 816.0798758120791 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.415460582524272 > 1.8392393023130715. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -74.61505096644541 (T) = (0 -2427494.8767737118) / Math.Sqrt((323923608.37270844 / (32)) + (296898246.4512639 / (4))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (4) - 2, .025) and -0.3933303824777931 = (1742224.893177768 - 2427494.8767737118) / 1742224.893177768 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.2846865454545457 > 1.7570382990147082. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -44.86554147093042 (T) = (0 -2281122.8476508204) / Math.Sqrt((6121350019.956217 / (32)) + (26362135.53215247 / (4))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (4) - 2, .025) and -0.3825022860354526 = (1649995.7147935766 - 2281122.8476508204) / 1649995.7147935766 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 688.7860936339338 > 640.5068893931114. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -19.714809983865944 (T) = (0 -693.3010685495831) / Math.Sqrt((415.262503892004 / (31)) + (9.781134450586094 / (5))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (5) - 2, .025) and -0.12538688323962394 = (616.0557572466049 - 693.3010685495831) / 616.0557572466049 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 852.2916097138691 > 823.448612216711. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -12.252480897468942 (T) = (0 -868.961286914838) / Math.Sqrt((674.7271062598871 / (32)) + (130.39023230815414 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.10721553680619643 = (784.8167389534548 - 868.961286914838) / 784.8167389534548 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.004850787766451 > 2.353762709459459. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -79.8434991165326 (T) = (0 -2998308.751621872) / Math.Sqrt((2545433805.7557793 / (32)) + (106038731.82518518 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.36480621667235413 = (2196875.0691451966 - 2998308.751621872) / 2196875.0691451966 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 999.6178122589354 > 969.0094455313405. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -34.32117893322841 (T) = (0 -1004.5701016187271) / Math.Sqrt((81.57039517391033 / (32)) + (15.575711823555888 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.08850822114513077 = (922.8870137167185 - 1004.5701016187271) / 922.8870137167185 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IntObj - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/PerfLabTests.CastingPerf2.CastingPerf.IntObj.html>) 111.41 μs 167.07 μs 1.50 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.CastingPerf2.CastingPerf.IntObj ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 167.06675056990883 > 116.9493656659165. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -2395.4141351212506 (T) = (0 -167110.62512664642) / Math.Sqrt((1493.0841279847477 / (32)) + (2469.6480345619866 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.49988119598946923 = (111415.90785555773 - 167110.62512664642) / 111415.90785555773 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20AllEscaped).html>) 57.66 ms 65.28 ms 1.13 0.13 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20AllEscaped).html>) 57.84 ms 65.03 ms 1.12 0.08 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20AllEscaped).html>) 57.02 ms 64.87 ms 1.14 0.27 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: True, Escaped: AllEscaped) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 65.27616166666667 > 60.932350500000005. IsChangePoint: Marked as a change because one of 3/29/2023 6:23:42 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -18.6819149672443 (T) = (0 -65306889.10256411) / Math.Sqrt((3557848081438.1514 / (32)) + (63492322445.20079 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.11195753507049146 = (58731459.64915292 - 65306889.10256411) / 58731459.64915292 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: False, Escaped: AllEscaped) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 65.03360833333333 > 60.742794. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -23.466284109363542 (T) = (0 -65071896.38095238) / Math.Sqrt((2483618476648.589 / (32)) + (13096555367.471935 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.1137680974310779 = (58424995.76980311 - 65071896.38095238) / 58424995.76980311 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: False, Escaped: AllEscaped) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 64.87416666666667 > 59.89409596153846. IsChangePoint: Marked as a change because one of 3/7/2023 7:00:18 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -6.636003985283821 (T) = (0 -66659244) / Math.Sqrt((5138871665394.493 / (32)) + (6211510472852.222 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.13367385713058583 = (58799313.0305744 - 66659244) / 58799313.0305744 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString_Flags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Enum.ToString_Flags(value%3a%2032).html>) 20.26 ns 22.02 ns 1.09 0.10 False
[ToString_NonFlags_Small - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Enum.ToString_NonFlags_Small(value%3a%20-1).html>) 26.60 ns 31.47 ns 1.18 0.15 False
[ToString_NonFlags_Large - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Enum.ToString_NonFlags_Large(value%3a%2042).html>) 10.97 ns 12.08 ns 1.10 0.15 False
[ToString_Flags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Enum.ToString_Flags(value%3a%2036).html>) 21.37 ns 23.53 ns 1.10 0.08 False
[ToString_Format_Flags_Large - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Enum.ToString_Format_Flags_Large(value%3a%20All%2c%20format%3a%20%22d%22).html>) 18.20 ns 20.43 ns 1.12 0.12 False
[ToString_NonFlags_Large - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Enum.ToString_NonFlags_Large(value%3a%20Format).html>) 10.33 ns 11.80 ns 1.14 0.12 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Enum.ToString_Flags(value: 32) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.02238821756277 > 21.284938856673747. IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 2/23/2023 8:54:44 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -6.888413011848785 (T) = (0 -22.44697506748299) / Math.Sqrt((0.13827947252397543 / (32)) + (0.364140293855637 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.09317913193538847 = (20.533665903173958 - 22.44697506748299) / 20.533665903173958 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_Enum.ToString_NonFlags_Small(value: -1) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.466731533498876 > 27.804862436602598. IsChangePoint: Marked as a change because one of 3/17/2023 10:06:19 AM, 3/25/2023 12:18:49 AM, 4/5/2023 6:15:03 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -23.53412815712049 (T) = (0 -30.405230249748882) / Math.Sqrt((0.12255110196343084 / (21)) + (0.32737321065840036 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.1435316753360394 = (26.58888328634532 - 30.405230249748882) / 26.58888328634532 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_Enum.ToString_NonFlags_Large(value: 42) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.080455563054239 > 11.517148864691475. IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/28/2023 10:21:25 PM, 3/25/2023 12:18:49 AM, 3/31/2023 8:05:26 PM, 4/5/2023 6:15:03 PM, 4/9/2023 11:04:55 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -22.87891462206088 (T) = (0 -12.585450322938796) / Math.Sqrt((0.0002862176656798716 / (20)) + (0.08372796915355572 / (17))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (17) - 2, .025) and -0.14647832517206086 = (10.977486487631591 - 12.585450322938796) / 10.977486487631591 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_Enum.ToString_Flags(value: 36) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.533132191447102 > 22.423954023771167. IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -21.000459776006714 (T) = (0 -23.662539407874373) / Math.Sqrt((0.011723627005496996 / (32)) + (0.05916775781051082 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.10868067613362639 = (21.342970899785385 - 23.662539407874373) / 21.342970899785385 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_Enum.ToString_Format_Flags_Large(value: All, format: "d") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.43269485489117 > 19.3321533084613. IsChangePoint: Marked as a change because one of 2/4/2023 4:58:40 AM, 2/14/2023 11:21:41 AM, 2/15/2023 3:19:23 AM, 3/15/2023 5:33:41 AM, 3/25/2023 12:18:49 AM, 4/9/2023 11:04:55 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -9.177943276825385 (T) = (0 -20.000351459146277) / Math.Sqrt((0.010310861207640494 / (20)) + (0.5030109445177491 / (17))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (17) - 2, .025) and -0.08650790181949729 = (18.407920849588958 - 20.000351459146277) / 18.407920849588958 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_Enum.ToString_NonFlags_Large(value: Format) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.795520430287015 > 10.840665210240592. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 3/31/2023 8:05:26 PM, 4/5/2023 6:15:03 PM, 4/9/2023 11:04:55 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -8.555631662732608 (T) = (0 -12.098562416279623) / Math.Sqrt((0.5583683755308672 / (20)) + (0.21490909364804503 / (17))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (17) - 2, .025) and -0.16607063151531387 = (10.375497066209007 - 12.098562416279623) / 10.375497066209007 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArrayOr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.Tests.Perf_BitArray.BitArrayOr(Size%3a%20512).html>) 10.17 ns 13.26 ns 1.30 0.16 False
[BitArrayAnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size%3a%20512).html>) 10.21 ns 13.27 ns 1.30 0.15 False
[BitArrayXor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.Tests.Perf_BitArray.BitArrayXor(Size%3a%20512).html>) 11.09 ns 13.28 ns 1.20 0.16 False
[BitArrayNot - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.Tests.Perf_BitArray.BitArrayNot(Size%3a%20512).html>) 7.35 ns 12.37 ns 1.68 0.09 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_BitArray.BitArrayOr(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.258290928339537 > 10.708245939186488. IsChangePoint: Marked as a change because one of 2/7/2023 2:48:42 AM, 3/3/2023 11:43:56 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -5.758779981705897 (T) = (0 -12.821558312523257) / Math.Sqrt((0.05269811776218099 / (32)) + (0.9834212105616709 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.25003961068666336 = (10.256921623051772 - 12.821558312523257) / 10.256921623051772 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Collections.Tests.Perf_BitArray.BitArrayAnd(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.27216463427472 > 10.720023190124522. IsChangePoint: Marked as a change because one of 2/7/2023 8:22:47 AM, 3/3/2023 11:43:56 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -5.86701365115458 (T) = (0 -12.83692960145809) / Math.Sqrt((0.04271926931425503 / (32)) + (0.9567778235409867 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.2509781129352296 = (10.261514145389954 - 12.83692960145809) / 10.261514145389954 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Collections.Tests.Perf_BitArray.BitArrayXor(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.275923870933553 > 10.712044068455468. IsChangePoint: Marked as a change because one of 2/6/2023 9:02:07 PM, 3/3/2023 11:43:56 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -44.53795757940713 (T) = (0 -13.269608172979854) / Math.Sqrt((0.1315302226101734 / (32)) + (0.00013909517594472087 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.2753653754575702 = (10.404554199394852 - 13.269608172979854) / 10.404554199394852 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Collections.Tests.Perf_BitArray.BitArrayNot(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.37262585211462 > 7.71451204766702. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -62.085894731541025 (T) = (0 -12.357110693446064) / Math.Sqrt((0.17594889521191662 / (32)) + (0.0017234859549671295 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.623531485438496 = (7.6112541113476215 - 12.357110693446064) / 7.6112541113476215 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 692.46 ns 757.34 ns 1.09 0.04 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToString(Mode%3a%20Reflection).html>) 718.60 ns 819.37 ns 1.14 0.04 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToWriter(Mode%3a%20SourceGen).html>) 308.15 ns 343.48 ns 1.11 0.06 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToStream(Mode%3a%20Reflection).html>) 785.52 ns 879.24 ns 1.12 0.06 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 394.09 ns 433.12 ns 1.10 0.07 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeObjectProperty(Mode%3a%20Reflection).html>) 964.37 ns 1.04 μs 1.08 0.08 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToWriter(Mode%3a%20Reflection).html>) 636.33 ns 705.18 ns 1.11 0.06 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 757.3395537991107 > 725.0587199723154. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -8.00852945389047 (T) = (0 -766.7801009593353) / Math.Sqrt((70.77201827600845 / (32)) + (348.49027284735075 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.09717425758913148 = (698.8681111095465 - 766.7801009593353) / 698.8681111095465 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 819.3675267463816 > 761.345346618826. IsChangePoint: Marked as a change because one of 3/10/2023 8:23:08 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -17.252685061450357 (T) = (0 -816.3416183575482) / Math.Sqrt((231.42879790062847 / (32)) + (62.378713771072846 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.1035359931352988 = (739.7507860511269 - 816.3416183575482) / 739.7507860511269 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 343.4815613448336 > 326.2885969176305. IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -9.605356221626206 (T) = (0 -347.78798987919237) / Math.Sqrt((144.37200198687793 / (32)) + (29.91303106994909 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.09826065034388956 = (316.6716296084106 - 347.78798987919237) / 316.6716296084106 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 879.2416875216261 > 815.2898151174146. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -16.84472710076467 (T) = (0 -869.631315408124) / Math.Sqrt((77.5101412410777 / (21)) + (356.2093516122238 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.10948447871113307 = (783.8156658291949 - 869.631315408124) / 783.8156658291949 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 433.12129556896497 > 413.58264137146244. IsChangePoint: Marked as a change because one of 3/20/2023 4:51:16 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -13.49341114251368 (T) = (0 -427.6461593675011) / Math.Sqrt((59.35843974435732 / (32)) + (17.393410713425958 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.07859738963655488 = (396.4835845853485 - 427.6461593675011) / 396.4835845853485 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0448431737410315 > 1015.280423587859. IsChangePoint: Marked as a change because one of 2/14/2023 10:45:34 AM, 3/27/2023 11:25:56 PM, 4/4/2023 11:36:00 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -16.595334032895817 (T) = (0 -1054.0687338492685) / Math.Sqrt((127.63895221622514 / (31)) + (122.64448038965662 / (5))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (5) - 2, .025) and -0.0920198980025846 = (965.246819931823 - 1054.0687338492685) / 965.246819931823 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 705.1753673182366 > 668.1898057352752. IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -7.383901312900456 (T) = (0 -688.18398037476) / Math.Sqrt((226.20846171277276 / (21)) + (514.0375126882889 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.07558779298256597 = (639.8213003760956 - 688.18398037476) / 639.8213003760956 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Double.ToStringWithFormat(value%3a%20-1.7976931348623157E%2b308%2c%20format%3a%20%22G17%22).html>) 174.05 ns 203.79 ns 1.17 0.03 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Double.ToStringWithFormat(value%3a%20-1.7976931348623157E%2b308%2c%20format%3a%20%22E%22).html>) 133.41 ns 149.47 ns 1.12 0.02 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Double.ToStringWithFormat(value%3a%201.7976931348623157E%2b308%2c%20format%3a%20%22G17%22).html>) 171.47 ns 202.99 ns 1.18 0.06 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Double.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22E%22).html>) 133.75 ns 142.39 ns 1.06 0.02 False
[ToStringWithCultureInfo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Double.ToStringWithCultureInfo(value%3a%20-1.7976931348623157E%2b308%2c%20culture%3a%20zh).html>) 184.83 ns 198.31 ns 1.07 0.02 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 203.79245453993389 > 183.05059472235752. IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 2/23/2023 8:54:44 PM, 3/6/2023 11:14:52 AM, 3/16/2023 8:14:39 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -6.849964473450428 (T) = (0 -198.33984876852497) / Math.Sqrt((1.576333659739453 / (32)) + (63.87462628936584 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.14113334549743775 = (173.80952852803154 - 198.33984876852497) / 173.80952852803154 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 149.46613180913218 > 140.44113852650585. IsChangePoint: Marked as a change because one of 1/29/2023 5:12:48 PM, 2/4/2023 4:58:40 AM, 3/24/2023 7:28:10 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -8.470300660324138 (T) = (0 -146.22825567984052) / Math.Sqrt((2.619213934972474 / (21)) + (20.821996543224078 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.0743145516427656 = (136.11307363959526 - 146.22825567984052) / 136.11307363959526 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 202.98985735754118 > 179.76915923592097. IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 2/23/2023 8:54:44 PM, 3/6/2023 11:14:52 AM, 3/16/2023 8:14:39 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -7.19667397192826 (T) = (0 -200.13025571694521) / Math.Sqrt((6.482284916367706 / (32)) + (62.204557109545384 / (4))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (4) - 2, .025) and -0.166491510405247 = (171.56597706177786 - 200.13025571694521) / 171.56597706177786 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 142.3949147077147 > 139.59359309657484. IsChangePoint: Marked as a change because one of 2/9/2023 4:10:34 PM, 2/17/2023 9:53:52 PM, 3/14/2023 8:42:09 PM, 3/23/2023 6:11:55 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -10.535386307266908 (T) = (0 -143.44228683821083) / Math.Sqrt((2.443691027152454 / (21)) + (10.227095855604142 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.06819595154721729 = (134.28461943751364 - 143.44228683821083) / 134.28461943751364 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_Double.ToStringWithCultureInfo(value: -1.7976931348623157E+308, culture: zh) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 198.31405126992263 > 194.15453613009558. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/17/2023 1:30:52 PM, 4/11/2023 5:52:15 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -10.548941775784728 (T) = (0 -198.1351510315822) / Math.Sqrt((28.72450092921426 / (32)) + (3.203999753157747 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.07070629623672567 = (185.0508881174786 - 198.1351510315822) / 185.0508881174786 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cw%2b%5c%5cs%2bHolmes%22%2c%20Options%3a%20None).html>) 11.07 ms 13.11 ms 1.18 0.07 False
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22%5c%5cw%2b%5c%5cs%2bHolmes%5c%5cs%2b%5c%5cw%2b%22%2c%20Options%3a%20None).html>) 11.14 ms 13.19 ms 1.18 0.07 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+\\s+Holmes", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.114604444444444 > 11.682123785714285. IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 2/24/2023 8:06:06 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -12.072438859288473 (T) = (0 -12970394.601889338) / Math.Sqrt((97069415818.5781 / (32)) + (96694772757.12627 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.1617353979939917 = (11164671.941894654 - 12970394.601889338) / 11164671.941894654 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\\w+\\s+Holmes\\s+\\w+", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.18703777777778 > 11.818304729166664. IsChangePoint: Marked as a change because one of 2/23/2023 8:54:44 PM, 2/24/2023 8:06:06 PM, 4/10/2023 1:33:42 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -8.575469790278731 (T) = (0 -12857579.588606752) / Math.Sqrt((41049573368.273186 / (21)) + (509072937454.0615 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.13968898305606495 = (11281656.469231876 - 12857579.588606752) / 11281656.469231876 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ScaleB - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.MathBenchmarks.Double.ScaleB.html>) 11.42 μs 13.38 μs 1.17 0.10 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.MathBenchmarks.Double.ScaleB ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.383365566903132 > 11.993649825246713. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/22/2023 7:51:01 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -44.042259799514845 (T) = (0 -13384.85493809717) / Math.Sqrt((58333.38195517165 / (32)) + (27.04082550394611 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.16373296294284717 = (11501.654902211894 - 13384.85493809717) / 11501.654902211894 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Tests.Perf_Encoding.GetBytes(size%3a%2016%2c%20encName%3a%20%22utf-8%22).html>) 17.36 ns 19.56 ns 1.13 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "utf-8") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.560966852646292 > 18.088014241861995. IsChangePoint: Marked as a change because one of 2/4/2023 4:58:40 AM, 4/5/2023 6:15:03 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -14.17583461446329 (T) = (0 -19.868352999818736) / Math.Sqrt((0.15120582985324443 / (32)) + (0.13608458211505978 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.14615429388538795 = (17.334797859079096 - 19.868352999818736) / 17.334797859079096 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Exp - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.MathBenchmarks.Single.Exp.html>) 19.50 μs 20.90 μs 1.07 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.MathBenchmarks.Single.Exp ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.904383355614975 > 20.471861128428927. IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/6/2023 1:55:21 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -333.6723717275789 (T) = (0 -20898.43832760377) / Math.Sqrt((70.04550897889082 / (32)) + (76.47561052115962 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.07153767270780323 = (19503.22313427663 - 20898.43832760377) / 19503.22313427663 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in SciMark2.kernel

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[benchMonteCarlo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/SciMark2.kernel.benchMonteCarlo.html>) 732.24 ms 777.82 ms 1.06 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SciMark2.kernel*'
### Payloads [Baseline]() [Compare]() ### Histogram #### SciMark2.kernel.benchMonteCarlo ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 777.8181666666666 > 768.5184775. IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -114.90253754577145 (T) = (0 -777114192.3809524) / Math.Sqrt((262248498091.99866 / (32)) + (718525163175.9836 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.061150658860471566 = (732331630.6616302 - 777114192.3809524) / 732331630.6616302 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Int64.TryFormat(value%3a%20-9223372036854775808).html>) 22.08 ns 24.71 ns 1.12 0.07 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.706838872271025 > 23.394319555584357. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -13.049306127531848 (T) = (0 -24.692222098850223) / Math.Sqrt((0.13954619832780385 / (32)) + (0.12063363183917049 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.09793338067154078 = (22.48972709414068 - 24.692222098850223) / 22.48972709414068 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IntObj - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/PerfLabTests.CastingPerf.IntObj.html>) 278.63 μs 306.55 μs 1.10 0.04 False
[FooObjIsFoo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/PerfLabTests.CastingPerf.FooObjIsFoo.html>) 278.39 μs 306.20 μs 1.10 0.04 False
[ScalarValueTypeObj - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/PerfLabTests.CastingPerf.ScalarValueTypeObj.html>) 278.56 μs 306.52 μs 1.10 0.03 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.CastingPerf.IntObj ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 306.55254084967333 > 292.4503878348214. IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -476.6335632313997 (T) = (0 -306402.8088235295) / Math.Sqrt((8536.57366676373 / (32)) + (15761.038402698436 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.1000589240242442 = (278533.0877573761 - 306402.8088235295) / 278533.0877573761 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### PerfLabTests.CastingPerf.FooObjIsFoo ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 306.20449346405235 > 292.38102734374996. IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -774.5680404526934 (T) = (0 -306298.3055016879) / Math.Sqrt((8736.354219272609 / (32)) + (5069.177256608562 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.09976501084180482 = (278512.5026547578 - 306298.3055016879) / 278512.5026547578 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### PerfLabTests.CastingPerf.ScalarValueTypeObj ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 306.5174868024133 > 292.46149714543265. IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -628.2506891504948 (T) = (0 -306370.36656970484) / Math.Sqrt((11009.293554981532 / (32)) + (8100.395979288032 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.0999652735716006 = (278527.30802574934 - 306370.36656970484) / 278527.30802574934 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FrozenDictionaryOptimized - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.TryGetValueFalse(Int32%2c%20Int32).FrozenDictionaryOptimized(Size%3a%20512).html>) 2.02 μs 2.30 μs 1.14 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueFalse<Int32, Int32>.FrozenDictionaryOptimized(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.300556947720126 > 2.1204380998807664. IsChangePoint: Marked as a change because one of 4/5/2023 11:58:06 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -55.2613351997733 (T) = (0 -2299.91563076655) / Math.Sqrt((786.0516722913036 / (32)) + (0.9606200741641446 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.1357836266821238 = (2024.9593115592925 - 2299.91563076655) / 2024.9593115592925 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToWriter(Mode%3a%20SourceGen).html>) 224.94 ns 240.78 ns 1.07 0.11 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 240.7808487615971 > 231.169182228494. IsChangePoint: Marked as a change because one of 3/17/2023 10:06:19 AM, 3/17/2023 9:25:32 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -7.518627266223622 (T) = (0 -242.18997142617135) / Math.Sqrt((158.30693083040185 / (32)) + (9.468625034735519 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.0883717894657425 = (222.5250358106553 - 242.18997142617135) / 222.5250358106553 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<StructRecord>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToWriter(Mode%3a%20SourceGen).html>) 214.70 ns 235.07 ns 1.09 0.14 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 235.0658670964751 > 227.2429716902138. IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -9.712122322353553 (T) = (0 -231.72080701297403) / Math.Sqrt((65.48327541022006 / (32)) + (6.948838106821249 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.08423796805423553 = (213.71766516239813 - 231.72080701297403) / 213.71766516239813 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Char).Reverse(Size%3a%20512).html>) 16.08 ns 21.38 ns 1.33 0.12 False
[EndsWith - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Char).EndsWith(Size%3a%20512).html>) 11.33 ns 12.40 ns 1.09 0.11 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Char>.Reverse(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.381525719804003 > 16.8818616799277. IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -23.181141106749667 (T) = (0 -21.738999678689165) / Math.Sqrt((0.1166573495073537 / (31)) + (0.273651686960397 / (5))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (5) - 2, .025) and -0.347524735906971 = (16.132542208256694 - 21.738999678689165) / 16.132542208256694 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Memory.Span<Char>.EndsWith(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.404484176652609 > 11.919288855380916. IsChangePoint: Marked as a change because one of 2/1/2023 6:25:50 PM, 3/3/2023 11:43:56 AM, 3/10/2023 4:14:40 PM, 4/3/2023 9:14:01 PM, 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -21.35772311569363 (T) = (0 -12.449429052856509) / Math.Sqrt((0.06988851517842119 / (32)) + (0.000682515117413055 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.09008635081187599 = (11.420589794179522 - 12.449429052856509) / 11.420589794179522 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Array.Reverse.html>) 16.09 ns 21.55 ns 1.34 0.01 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Array.Reverse ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.552113254768596 > 17.03112065805194. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 7:36:21 AM falls between 4/9/2023 7:17:29 PM and 4/18/2023 7:36:21 AM. IsRegressionStdDev: Marked as regression because -28.392909011090723 (T) = (0 -21.332357042937126) / Math.Sqrt((0.01979169888840168 / (32)) + (0.16122769657083175 / (5))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (5) - 2, .025) and -0.3180190565534204 = (16.18516586453658 - 21.332357042937126) / 16.18516586453658 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)