search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 24 Regressions on 6/22/2023 1:44:15 PM #19291

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Doubles

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteDoubles - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted%3a%20True%2c%20SkipValidation%3a%20True).html>) 16.20 ms 17.65 ms 1.09 0.02 False
[WriteDoubles - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 16.08 ms 17.73 ms 1.10 0.01 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Doubles*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.6497306125 > 16.94779835. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -11.168937933839342 (T) = (0 -17802798.612156596) / Math.Sqrt((51234499644.3203 / (28)) + (206714853794.88986 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.08792517020512783 = (16363991.844034536 - 17802798.612156596) / 16363991.844034536 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.7300947 > 16.97934174038462. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -21.479619399529803 (T) = (0 -18034461.10357143) / Math.Sqrt((46710982525.11407 / (27)) + (70295357432.84024 / (14))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (14) - 2, .025) and -0.10847721477783127 = (16269582.146698453 - 18034461.10357143) / 16269582.146698453 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FooObjIsFoo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/PerfLabTests.CastingPerf2.CastingPerf.FooObjIsFoo.html>) 521.49 μs 620.65 μs 1.19 0.04 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### PerfLabTests.CastingPerf2.CastingPerf.FooObjIsFoo ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 620.6517694444444 > 543.9732582333671. IsChangePoint: Marked as a change because one of 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -8.53815173605662 (T) = (0 -605635.0734937907) / Math.Sqrt((10558331.173463888 / (28)) + (1482571917.3927193 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.17004832785750162 = (517615.4344007148 - 605635.0734937907) / 517615.4344007148 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Perf_Utf8Encoding.GetBytes(Input%3a%20Cyrillic).html>) 99.62 μs 109.94 μs 1.10 0.06 False
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Perf_Utf8Encoding.GetString(Input%3a%20Greek).html>) 268.24 μs 283.44 μs 1.06 0.02 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 109.93887546828171 > 104.99624643501029. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 6/5/2023 4:27:01 PM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -46.334847622640595 (T) = (0 -110006.25352950217) / Math.Sqrt((314557.60580416914 / (28)) + (472396.6628678858 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.09808948788809257 = (100179.68001958782 - 110006.25352950217) / 100179.68001958782 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Perf_Utf8Encoding.GetString(Input: Greek) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 283.4438588598902 > 281.08420727723677. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -30.146475621686214 (T) = (0 -283080.21459449275) / Math.Sqrt((6512071.688655229 / (28)) + (775432.3290941997 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.060610897604399964 = (266902.9850946144 - 283080.21459449275) / 266902.9850946144 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.AddGivenSize(String).IDictionary(Size%3a%20512).html>) 10.28 μs 10.98 μs 1.07 0.04 False
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.AddGivenSize(String).Dictionary(Size%3a%20512).html>) 9.59 μs 10.46 μs 1.09 0.03 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Collections.AddGivenSize<String>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.978804947368419 > 10.787768285241476. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 6/5/2023 4:27:01 PM, 6/9/2023 2:21:24 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -22.07361735786894 (T) = (0 -11040.776468299593) / Math.Sqrt((8049.730986111176 / (28)) + (13333.395888669784 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.07572968303432281 = (10263.523115915841 - 11040.776468299593) / 10263.523115915841 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Collections.AddGivenSize<String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.460063626201654 > 10.09869509948163. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 6/5/2023 4:27:01 PM, 6/9/2023 2:21:24 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -9.751756845746915 (T) = (0 -10363.054192148382) / Math.Sqrt((10274.086801870477 / (28)) + (71653.67537028252 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.07491329092175698 = (9640.828036707855 - 10363.054192148382) / 9640.828036707855 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Sort(Int32).Array(Size%3a%20512).html>) 3.31 μs 5.01 μs 1.51 0.46 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Collections.Sort<Int32>.Array(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.011824340000001 > 3.6185184187500004. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -8.654946360103027 (T) = (0 -4862.555918571428) / Math.Sqrt((31964.27629407413 / (28)) + (388278.6047646556 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.4336075256029058 = (3391.8320263605433 - 4862.555918571428) / 3391.8320263605433 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Int32.ToString(value%3a%202147483647).html>) 12.12 ns 14.00 ns 1.16 0.11 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Tests.Perf_Int32.ToString(value: 2147483647) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.995972922921606 > 12.748953832131747. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -10.249237387238233 (T) = (0 -13.454748906095618) / Math.Sqrt((0.09690927555723083 / (28)) + (0.2425814542765987 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.12338232462093465 = (11.976998935456532 - 13.454748906095618) / 11.976998935456532 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Combine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.Perf_Path.Combine.html>) 1.58 ns 2.69 ns 1.70 0.22 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.IO.Tests.Perf_Path.Combine ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.6878766281604736 > 1.657866176126252. IsChangePoint: Marked as a change because one of 3/28/2023 3:45:48 AM, 4/12/2023 4:44:56 PM, 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -23.86144949699691 (T) = (0 -2.7445394663235083) / Math.Sqrt((0.03331296836266951 / (28)) + (0.012399949166552562 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.6558523638732286 = (1.6574783635321906 - 2.7445394663235083) / 1.6574783635321906 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20DeepTree).html>) 6.97 μs 7.75 μs 1.11 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Reader*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: DeepTree) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.749397356286187 > 7.28303320476574. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -12.120756699758482 (T) = (0 -7543.180785064028) / Math.Sqrt((2305.6523628287564 / (28)) + (32250.003193527773 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.08517297067235775 = (6951.1322055786 - 7543.180785064028) / 6951.1322055786 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString4 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Version.ToString4.html>) 27.97 ns 31.27 ns 1.12 0.06 False
[TryFormat2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Version.TryFormat2.html>) 7.63 ns 9.26 ns 1.21 0.17 False
[Parse2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Version.Parse2.html>) 22.01 ns 24.74 ns 1.12 0.09 False
[Parse4 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Version.Parse4.html>) 36.97 ns 44.66 ns 1.21 0.18 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Tests.Perf_Version.ToString4 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.274995029803648 > 29.37081472767498. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 6/5/2023 4:27:01 PM, 6/9/2023 2:21:24 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -20.337614459832164 (T) = (0 -31.182623624841124) / Math.Sqrt((0.21251053567551953 / (28)) + (0.24106603107163796 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.1144893149790035 = (27.97929348064552 - 31.182623624841124) / 27.97929348064552 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Version.TryFormat2 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.264781105221273 > 8.00402212572497. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/25/2023 6:46:56 PM, 5/19/2023 3:32:16 PM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -14.146534049928981 (T) = (0 -9.534548443054522) / Math.Sqrt((0.077714649533679 / (28)) + (0.18432045277512693 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.23051441034116826 = (7.748424856244475 - 9.534548443054522) / 7.748424856244475 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Version.Parse2 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.7446617367213 > 23.105160917338928. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -19.75617930599609 (T) = (0 -25.3854332398892) / Math.Sqrt((0.1235826499536045 / (28)) + (0.3424499720453572 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.15239716263141187 = (22.02837186957618 - 25.3854332398892) / 22.02837186957618 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Version.Parse4 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 44.65566299811358 > 38.52978245249965. IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -18.64163852718633 (T) = (0 -44.3533316469176) / Math.Sqrt((1.5254349711328246 / (28)) + (1.2263766730733492 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.18824707750059588 = (37.32669112909757 - 44.3533316469176) / 37.32669112909757 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryAddGiventSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.TryAddGiventSize(String).Dictionary(Count%3a%20512).html>) 9.48 μs 10.36 μs 1.09 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize&lt;String&gt;*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Collections.TryAddGiventSize<String>.Dictionary(Count: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.361296336309522 > 10.087158895453975. IsChangePoint: Marked as a change because one of 6/5/2023 4:27:01 PM, 6/9/2023 2:21:24 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -24.97312708451118 (T) = (0 -10320.153817188022) / Math.Sqrt((7351.070850283668 / (28)) + (8203.133388252369 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.07583173209271667 = (9592.721156414651 - 10320.153817188022) / 9592.721156414651 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 416.17 ns 494.20 ns 1.19 0.20 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 494.2039145082147 > 440.8692919394976. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -6.5233781502635715 (T) = (0 -583.3567150696538) / Math.Sqrt((544.392934222713 / (28)) + (7412.182435272 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.354988550968891 = (430.5251986465286 - 583.3567150696538) / 430.5251986465286 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.CtorFromCollection(Int32).ImmutableSortedSet(Size%3a%20512).html>) 12.25 μs 15.85 μs 1.29 0.09 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Collections.CtorFromCollection<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.84591804425229 > 12.779532324431637. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -8.76647794971968 (T) = (0 -14539.715693672024) / Math.Sqrt((33779.548954962585 / (28)) + (1020540.13197673 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.19635645961752568 = (12153.330704061531 - 14539.715693672024) / 12153.330704061531 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString_Flags - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Enum.ToString_Flags(value%3a%2036).html>) 18.30 ns 21.11 ns 1.15 0.11 False
[ToString_NonFlags_Large - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Enum.ToString_NonFlags_Large(value%3a%2042).html>) 10.81 ns 13.00 ns 1.20 0.25 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Tests.Perf_Enum.ToString_Flags(value: 36) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.111167468565803 > 19.34001578438359. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -10.597438331620632 (T) = (0 -20.40364090973932) / Math.Sqrt((0.6083611491001522 / (28)) + (0.07271097823924952 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.09315804494913528 = (18.664859124454143 - 20.40364090973932) / 18.664859124454143 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Enum.ToString_NonFlags_Large(value: 42) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.001551362277302 > 11.445025024604352. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -11.843563859077982 (T) = (0 -12.753887783675996) / Math.Sqrt((0.09896642871115811 / (28)) + (0.2800666706241629 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.1661457394021392 = (10.936787189407966 - 12.753887783675996) / 10.936787189407966 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Int16.ToString(value%3a%2032767).html>) 9.63 ns 11.39 ns 1.18 0.06 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Tests.Perf_Int16.ToString(value: 32767) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.390289306601302 > 10.110101991930222. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -12.211242952225726 (T) = (0 -11.022729638547531) / Math.Sqrt((0.0310654215190253 / (28)) + (0.14504578503192225 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.13461691908129275 = (9.714935017426598 - 11.022729638547531) / 9.714935017426598 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Equals_Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Perf_Ascii.Equals_Bytes(Size%3a%206).html>) 4.57 ns 5.70 ns 1.25 0.27 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Text.Perf_Ascii.Equals_Bytes(Size: 6) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.695653621656885 > 4.773122011939998. IsChangePoint: Marked as a change because one of 5/11/2023 1:19:27 PM, 6/13/2023 11:57:38 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -8.114127068301768 (T) = (0 -5.628386968360014) / Math.Sqrt((0.19534802527857487 / (28)) + (0.05969661351317465 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.18042370489037282 = (4.768107371143253 - 5.628386968360014) / 4.768107371143253 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[PropertyIndexer - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase%3a%20ObjectProperties).html>) 2.50 μs 2.64 μs 1.05 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase: ObjectProperties) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.6364926521678713 > 2.630939371423961. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -51.676295541277504 (T) = (0 -2638.5146055959804) / Math.Sqrt((68.58483554804566 / (28)) + (58.276302288337234 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.05303245587295505 = (2505.6346467580374 - 2638.5146055959804) / 2505.6346467580374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 86103fd589e2f335fb1a895027220e9f7960e973
Compare ba6da78bc8f4b2f2c59babe76b8accd78ece434e
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Get.GetString.html>) 2.55 μs 2.93 μs 1.15 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'
### Payloads [Baseline](<>) [Compare](<>) ### Histogram #### System.Text.Json.Tests.Perf_Get.GetString ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9257491273462697 > 2.7055080490219443. IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/23/2023 2:36:40 AM, 6/22/2023 7:55:14 AM, 6/26/2023 6:52:13 PM falls between 6/17/2023 6:37:57 PM and 6/26/2023 6:52:13 PM. IsRegressionStdDev: Marked as regression because -19.01564316301275 (T) = (0 -2859.9907185267257) / Math.Sqrt((3749.719628525603 / (28)) + (1296.7845620084809 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.11120343302733016 = (2573.777792186126 - 2859.9907185267257) / 2573.777792186126 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
EgorBo commented 1 year ago

Static PGO update