search

DrewScoggins / performance-2

This repo contains benchmarks used for testing the performance of all .NET Runtimes
MIT License
0 stars 0 forks source link

[Perf] Linux/x64: 29 Regressions on 10/5/2022 12:22:07 PM #8379

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff8ac3b98688eee989af9575f5f19e0572542ce8
Compare 2e698fba2213fdd193ddd01d145823c73263fbee
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt32.TryFormat(value%3a%200).html>) 232.98 ns 261.59 ns 1.12 0.16 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_UInt32.TryFormat(value: 0) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 261.5886741157346 > 226.82248733060476. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -13.586076292486315 (T) = (0 -299.94498739333613) / Math.Sqrt((53.4443231197033 / (22)) + (631.3152590989534 / (18))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (18) - 2, .025) and -0.38386124995257603 = (216.74498610580724 - 299.94498739333613) / 216.74498610580724 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff8ac3b98688eee989af9575f5f19e0572542ce8
Compare 246e6e0329b60c12d2de1378eca2eff55f5cc609
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FormatterInt32 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%20-2147483648).html>) 432.26 ns 507.75 ns 1.17 0.13 False
[FormatterUInt32 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value%3a%2012345).html>) 282.47 ns 354.87 ns 1.26 0.18 False
[FormatterInt64 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%209223372036854775807).html>) 758.22 ns 844.57 ns 1.11 0.06 False
[FormatterInt64 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%20-9223372036854775808).html>) 779.44 ns 881.86 ns 1.13 0.04 False
[FormatterInt64 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%2012345).html>) 306.02 ns 380.55 ns 1.24 0.16 False
[FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%2012345).html>) 279.16 ns 360.98 ns 1.29 0.19 False
[FormatterInt32 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%2012345).html>) 312.74 ns 378.91 ns 1.21 0.18 False
[FormatterUInt32 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value%3a%204294967295).html>) 438.25 ns 501.99 ns 1.15 0.12 False
[FormatterInt32 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%202147483647).html>) 444.56 ns 527.58 ns 1.19 0.15 False
[FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%2018446744073709551615).html>) 794.28 ns 877.89 ns 1.11 0.05 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 507.74788244068174 > 460.09868852704403. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -22.912729870458996 (T) = (0 -503.9070047744135) / Math.Sqrt((116.2094060434702 / (22)) + (44.08872895916805 / (17))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (17) - 2, .025) and -0.14627104124078671 = (439.605456863812 - 503.9070047744135) / 439.605456863812 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 354.87431125434415 > 296.51184960326. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -26.06600625881225 (T) = (0 -363.4783092393296) / Math.Sqrt((39.90868433099519 / (22)) + (129.46314510940627 / (18))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (18) - 2, .025) and -0.2742341438488032 = (285.25236982070606 - 363.4783092393296) / 285.25236982070606 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 844.5723042482836 > 807.8779602728275. IsChangePoint: Marked as a change because one of 10/5/2022 7:15:47 PM, 10/6/2022 8:29:49 AM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -2.548416266699156 (T) = (0 -854.7657749151025) / Math.Sqrt((11351.178204991018 / (28)) + (93.38270456506592 / (10))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (10) - 2, .025) and -0.06464179442598442 = (802.8670106605769 - 854.7657749151025) / 802.8670106605769 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: -9223372036854775808) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 881.8594994245137 > 820.5141544929428. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -16.44130801969484 (T) = (0 -875.6621054010808) / Math.Sqrt((123.32751054867164 / (22)) + (373.3206097207191 / (17))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (17) - 2, .025) and -0.10935982170927258 = (789.3400213934948 - 875.6621054010808) / 789.3400213934948 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 380.5521876180685 > 322.17497383629063. IsChangePoint: Marked as a change because one of 10/3/2022 11:00:05 AM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -22.696656103707912 (T) = (0 -387.43688649710674) / Math.Sqrt((96.64752595764901 / (19)) + (102.37038106402606 / (18))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (18) - 2, .025) and -0.23806255093919632 = (312.93805486903426 - 387.43688649710674) / 312.93805486903426 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 360.97934352133575 > 293.8561397549421. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -30.24069173314917 (T) = (0 -363.6056254280113) / Math.Sqrt((30.506192748258734 / (21)) + (103.80820007824185 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.2877840283369876 = (282.34984859810896 - 363.6056254280113) / 282.34984859810896 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 378.9057921637841 > 324.7048463545247. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -27.177465246081596 (T) = (0 -384.77070654765384) / Math.Sqrt((34.66143387763976 / (20)) + (109.04306361381163 / (19))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (19) - 2, .025) and -0.2392756715389596 = (310.48031958041844 - 384.77070654765384) / 310.48031958041844 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 4294967295) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 501.9913484353508 > 440.73793652636004. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -20.406598415063133 (T) = (0 -491.4518179095355) / Math.Sqrt((55.60139348618982 / (23)) + (191.3440546264381 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.17645386427498086 = (417.7399835499754 - 491.4518179095355) / 417.7399835499754 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 527.5821665650004 > 462.9273372553266. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -25.357602484752874 (T) = (0 -517.0198834905513) / Math.Sqrt((10.695020865001817 / (21)) + (128.81602380127225 / (15))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (15) - 2, .025) and -0.1736098225766202 = (440.53813588186557 - 517.0198834905513) / 440.53813588186557 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 877.8871549183825 > 838.9275913086306. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -19.35745940135511 (T) = (0 -880.8516368446544) / Math.Sqrt((202.18370115075612 / (20)) + (102.54375324081346 / (16))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (16) - 2, .025) and -0.09807505939487692 = (802.1779834705205 - 880.8516368446544) / 802.1779834705205 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff8ac3b98688eee989af9575f5f19e0572542ce8
Compare 2e698fba2213fdd193ddd01d145823c73263fbee
Diff Diff

Regressions in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TimeSpan_ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString.html>) 2.87 μs 3.09 μs 1.08 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 3.088151057637198 > 3.0124125766531855. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -20.07739915346219 (T) = (0 -3216.597108590859) / Math.Sqrt((1637.4264538768753 / (21)) + (3908.4412008290637 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.12010473799273455 = (2871.6931546554383 - 3216.597108590859) / 2871.6931546554383 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff8ac3b98688eee989af9575f5f19e0572542ce8
Compare 2e698fba2213fdd193ddd01d145823c73263fbee
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>) 69.77 ms 81.67 ms 1.17 0.06 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>) 51.01 ms 59.42 ms 1.17 0.06 False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>) 67.49 ms 72.88 ms 1.08 0.05 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>) 67.05 ms 74.89 ms 1.12 0.08 False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>) 48.57 ms 57.96 ms 1.19 0.07 False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>) 47.57 ms 56.89 ms 1.20 0.08 False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>) 51.36 ms 60.73 ms 1.18 0.04 False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>) 69.50 ms 76.94 ms 1.11 0.06 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 81.67121426785715 > 72.73897414740385. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -11.536158643138412 (T) = (0 -78227354.24973673) / Math.Sqrt((808005053214.2018 / (21)) + (8621414682390.3545 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.1261885824328034 = (69462038.12575443 - 78227354.24973673) / 69462038.12575443 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 59.42363326666666 > 53.471033722499996. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -18.465321700586415 (T) = (0 -58988581.3565873) / Math.Sqrt((846852412166.5149 / (20)) + (1952897131675.5154 / (15))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (15) - 2, .025) and -0.14945896181062088 = (51318562.31184525 - 58988581.3565873) / 51318562.31184525 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 72.87774996666667 > 69.803212465. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -22.575327747858243 (T) = (0 -74057538.34146826) / Math.Sqrt((703106771252.9565 / (21)) + (1333345524164.683 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.11107762004245941 = (66653793.58342055 - 74057538.34146826) / 66653793.58342055 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 74.89134998333333 > 69.9449077859375. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -16.924870358895724 (T) = (0 -74549782.43273811) / Math.Sqrt((2220876855537.6406 / (22)) + (1632979826349.9946 / (17))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (17) - 2, .025) and -0.11205912290002813 = (67037606.991908096 - 74549782.43273811) / 67037606.991908096 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 57.95608333333334 > 51.129461241250006. IsChangePoint: Marked as a change because one of 9/26/2022 7:45:50 PM, 9/27/2022 7:01:57 AM, 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -18.098503752948837 (T) = (0 -56424200.3350529) / Math.Sqrt((981518827677.1936 / (24)) + (2481320642992.229 / (18))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (18) - 2, .025) and -0.15688764840157077 = (48772411.40314027 - 56424200.3350529) / 48772411.40314027 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 56.8869999 > 50.58274750875. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -23.577034158237634 (T) = (0 -56266690.67556872) / Math.Sqrt((1184730220749.5608 / (22)) + (962424999725.2607 / (19))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (19) - 2, .025) and -0.15668278133466879 = (48644876.17827588 - 56266690.67556872) / 48644876.17827588 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 60.72808331666667 > 53.42524250875. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -12.405517908732161 (T) = (0 -59576382.660648145) / Math.Sqrt((532894089407.8096 / (21)) + (7983665190875.928 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.16630217483475046 = (51081429.792488664 - 59576382.660648145) / 51081429.792488664 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 76.93668341666667 > 72.08092968259615. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -18.047310705864785 (T) = (0 -77512878.95092055) / Math.Sqrt((1434668690539.4182 / (23)) + (2753914037860.708 / (19))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (19) - 2, .025) and -0.11858444797876531 = (69295509.24025722 - 77512878.95092055) / 69295509.24025722 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff8ac3b98688eee989af9575f5f19e0572542ce8
Compare 2e698fba2213fdd193ddd01d145823c73263fbee
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteDeepUtf16 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20False).html>) 175.76 ms 187.38 ms 1.07 0.04 False
[WriteDeepUtf8 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True).html>) 170.75 ms 193.70 ms 1.13 0.06 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 187.38133333333334 > 184.595844895. IsChangePoint: Marked as a change because one of 9/16/2022 2:24:22 AM, 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -8.40144766635979 (T) = (0 -189533196.89561406) / Math.Sqrt((8434783388642.789 / (22)) + (25026306037794.953 / (19))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (19) - 2, .025) and -0.06135158432091368 = (178577202.5929404 - 189533196.89561406) / 178577202.5929404 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 193.6960002857143 > 180.93449493000003. IsChangePoint: Marked as a change because one of 9/21/2022 8:46:36 AM, 10/3/2022 11:00:05 AM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -8.614534296067141 (T) = (0 -183722668.54196426) / Math.Sqrt((7853093096495.397 / (19)) + (16663085265393.008 / (16))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (16) - 2, .025) and -0.059944360116984445 = (173332370.50451127 - 183722668.54196426) / 173332370.50451127 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff8ac3b98688eee989af9575f5f19e0572542ce8
Compare 2e698fba2213fdd193ddd01d145823c73263fbee
Diff Diff

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryFormat4 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.TryFormat4.html>) 1.29 μs 1.62 μs 1.25 0.10 False
[ToString3 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.ToString3.html>) 1.41 μs 1.55 μs 1.10 0.05 False
[TryFormat2 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.TryFormat2.html>) 685.73 ns 809.44 ns 1.18 0.16 False
[ToString4 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.ToString4.html>) 1.70 μs 1.91 μs 1.12 0.06 False
[ToString2 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.ToString2.html>) 1.13 μs 1.21 μs 1.07 0.10 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Version.TryFormat4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.6176837589511373 > 1.357517720637363. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -41.25590212360766 (T) = (0 -1627.7274765470404) / Math.Sqrt((251.20315821255923 / (22)) + (971.53786369375 / (19))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (19) - 2, .025) and -0.2507169327692818 = (1301.4355478045688 - 1627.7274765470404) / 1301.4355478045688 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToString3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.549152212159864 > 1.4748857935865836. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -23.156371634647318 (T) = (0 -1650.21968297915) / Math.Sqrt((641.8483484290846 / (22)) + (1388.5710382594834 / (18))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (18) - 2, .025) and -0.16916361128457247 = (1411.4531679326183 - 1650.21968297915) / 1411.4531679326183 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.TryFormat2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 809.438448717556 > 714.5368452102958. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -19.238929120600208 (T) = (0 -849.2764847940267) / Math.Sqrt((458.7552232677654 / (22)) + (929.003585106115 / (19))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (19) - 2, .025) and -0.23333306955810917 = (688.6027025111018 - 849.2764847940267) / 688.6027025111018 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToString4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.9072442987152038 > 1.7882905074043267. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -20.396013748109663 (T) = (0 -2040.7226665797682) / Math.Sqrt((1408.2226861758886 / (21)) + (2990.492104090138 / (17))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (17) - 2, .025) and -0.18453819963752502 = (1722.8002163241676 - 2040.7226665797682) / 1722.8002163241676 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToString2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.2110805310915398 > 1.1357652538949639. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -20.889919596057577 (T) = (0 -1262.843547963167) / Math.Sqrt((317.87029482555636 / (23)) + (968.3797950042145 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.15744253462475874 = (1091.0637117484016 - 1262.843547963167) / 1091.0637117484016 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff8ac3b98688eee989af9575f5f19e0572542ce8
Compare 2e698fba2213fdd193ddd01d145823c73263fbee
Diff Diff

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_SByte.ToString(value%3a%20-128).html>) 401.37 ns 454.76 ns 1.13 0.12 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_SByte*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_SByte.ToString(value: -128) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 454.76045368657043 > 417.96424951449563. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -12.065822304269231 (T) = (0 -488.73797610749546) / Math.Sqrt((144.00060989087757 / (23)) + (826.6918044059481 / (18))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (18) - 2, .025) and -0.21705955331385915 = (401.57277002324156 - 488.73797610749546) / 401.57277002324156 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline ff8ac3b98688eee989af9575f5f19e0572542ce8
Compare 2e698fba2213fdd193ddd01d145823c73263fbee
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryFormat(value%3a%204).html>) 216.58 ns 267.26 ns 1.23 0.18 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Related Issues

Regressions

Improvements

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Int32.TryFormat(value: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 267.2581610851128 > 228.00087401588266. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM falls between 10/2/2022 4:42:49 PM and 10/11/2022 6:23:40 AM. IsRegressionStdDev: Marked as regression because -28.92860010926201 (T) = (0 -298.2640436239173) / Math.Sqrt((27.65275979703352 / (22)) + (115.05997370023223 / (18))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (18) - 2, .025) and -0.366579778037187 = (218.25585920224339 - 298.2640436239173) / 218.25585920224339 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)