search

DrewScoggins / performance-2

This repo contains benchmarks used for testing the performance of all .NET Runtimes
MIT License
0 stars 0 forks source link

[Perf] Linux/x64: 35 Regressions on 10/3/2022 11:31:18 PM #8357

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryFormat2 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Version.TryFormat2.html>) 55.19 ns 85.07 ns 1.54 0.01 False
[ToString4 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Version.ToString4.html>) 136.29 ns 195.80 ns 1.44 0.01 False
[TryFormat4 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Version.TryFormat4.html>) 106.83 ns 166.35 ns 1.56 0.02 False
[ToString3 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Version.ToString3.html>) 110.55 ns 154.94 ns 1.40 0.01 False
[TryFormatL - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Version.TryFormatL.html>) 252.83 ns 275.80 ns 1.09 0.00 False
[ToStringL - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Version.ToStringL.html>) 313.92 ns 337.43 ns 1.07 0.00 False
[TryFormat3 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Version.TryFormat3.html>) 81.49 ns 126.03 ns 1.55 0.01 False
[ToString2 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Version.ToString2.html>) 83.96 ns 113.56 ns 1.35 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Version.TryFormat2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 85.06604438429106 > 57.9486209776557. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -700.2099058846253 (T) = (0 -85.21060664807321) / Math.Sqrt((0.027550453019081336 / (24)) + (0.018547706308069643 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.5432034410675138 = (55.2167033719991 - 85.21060664807321) / 55.2167033719991 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToString4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 195.7965407134818 > 143.15986326306825. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -461.13700607066994 (T) = (0 -196.26440933874503) / Math.Sqrt((0.040845944328490896 / (26)) + (0.39804405973352686 / (26))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (26) - 2, .025) and -0.4394015154424207 = (136.35139829515904 - 196.26440933874503) / 136.35139829515904 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.TryFormat4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 166.34592661825354 > 112.26423445650099. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -687.397460576145 (T) = (0 -166.52301585222224) / Math.Sqrt((0.08475818897108844 / (26)) + (0.10609767093456918 / (25))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (25) - 2, .025) and -0.5566183161006325 = (106.97742287227258 - 166.52301585222224) / 106.97742287227258 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToString3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 154.93541945298676 > 115.67509999145493. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -479.54663100224445 (T) = (0 -155.32296058271876) / Math.Sqrt((0.03869125202185861 / (25)) + (0.19860176323339504 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.41107367947554974 = (110.07430925963219 - 155.32296058271876) / 110.07430925963219 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.TryFormatL ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 275.7955847250076 > 265.5627543357582. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -67.6661579125387 (T) = (0 -276.22430659561957) / Math.Sqrt((2.60064983141504 / (27)) + (0.4678854613300374 / (25))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (25) - 2, .025) and -0.09061442503819286 = (253.2740263232318 - 276.22430659561957) / 253.2740263232318 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToStringL ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 337.43014054891535 > 328.3304017307026. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -105.57873818741031 (T) = (0 -336.774960489643) / Math.Sqrt((0.8675116053919957 / (25)) + (0.45683588370191236 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.076689795932051 = (312.7873615613764 - 336.774960489643) / 312.7873615613764 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.TryFormat3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 126.02548941766591 > 85.52451577567675. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -621.8117208788102 (T) = (0 -126.32098647041482) / Math.Sqrt((0.03653243327788668 / (24)) + (0.09898388483399755 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.5493391037144759 = (81.53217469795058 - 126.32098647041482) / 81.53217469795058 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToString2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 113.55802380784812 > 88.02709000355662. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -403.37185162819947 (T) = (0 -113.86730956629367) / Math.Sqrt((0.03252875645087714 / (25)) + (0.11044053976099563 / (26))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (26) - 2, .025) and -0.3584764805306811 = (83.81986085015771 - 113.86730956629367) / 83.81986085015771 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_UInt32.ToString(value%3a%2012345).html>) 27.51 ns 36.71 ns 1.33 0.00 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_UInt32.TryFormat(value%3a%200).html>) 19.42 ns 32.96 ns 1.70 0.06 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_UInt32.ToString(value%3a%204294967295).html>) 49.55 ns 57.16 ns 1.15 0.00 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_UInt32.TryFormat(value%3a%2012345).html>) 31.97 ns 42.04 ns 1.31 0.01 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_UInt32.TryFormat(value%3a%204294967295).html>) 53.28 ns 59.59 ns 1.12 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_UInt32.ToString(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 36.70983664322332 > 29.12531292974337. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -150.520621708542 (T) = (0 -36.83763219740136) / Math.Sqrt((0.047205952920293635 / (25)) + (0.04815619250729026 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.32907344031448066 = (27.716777026772103 - 36.83763219740136) / 27.716777026772103 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.TryFormat(value: 0) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 32.95920927192129 > 19.610438631432004. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -51.39866993700678 (T) = (0 -33.922757759035626) / Math.Sqrt((0.12337536011335867 / (26)) + (2.1562929269133506 / (27))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (27) - 2, .025) and -0.7880243415123952 = (18.972201312618687 - 33.922757759035626) / 18.972201312618687 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.ToString(value: 4294967295) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 57.16491206064662 > 52.48767069502991. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -75.285347078734 (T) = (0 -56.91228872762064) / Math.Sqrt((0.17258785731647333 / (25)) + (0.055057256006570195 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.14298052429368238 = (49.79287705955465 - 56.91228872762064) / 49.79287705955465 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.TryFormat(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 42.03613865914321 > 33.50912074075017. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -266.203000832775 (T) = (0 -41.55768883372577) / Math.Sqrt((0.006540936236116493 / (23)) + (0.026555159111681385 / (26))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (26) - 2, .025) and -0.3011811716398065 = (31.938433893377752 - 41.55768883372577) / 31.938433893377752 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 59.588109294901074 > 55.32852767192792. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -106.96830600094441 (T) = (0 -59.78481806899) / Math.Sqrt((0.06442822718044487 / (25)) + (0.04292001783425693 / (26))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (26) - 2, .025) and -0.13165609695038274 = (52.829493191526765 - 59.78481806899) / 52.829493191526765 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IntegerFormatting - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/PerfLabTests.LowLevelPerf.IntegerFormatting.html>) 5.01 ms 5.81 ms 1.16 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### PerfLabTests.LowLevelPerf.IntegerFormatting ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 5.80709478611111 > 5.295717655754206. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -13.81506080949227 (T) = (0 -5854054.59641783) / Math.Sqrt((1220479459.2324476 / (23)) + (89550266630.4825 / (27))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (27) - 2, .025) and -0.15873741987335388 = (5052097.650438922 - 5854054.59641783) / 5052097.650438922 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Int16.ToString(value%3a%2032767).html>) 28.43 ns 37.04 ns 1.30 0.01 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Int16.ToString(value%3a%20-32768).html>) 37.66 ns 47.08 ns 1.25 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int16*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Int16.ToString(value: 32767) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 37.04384754159981 > 29.879832441491203. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -132.43372180251183 (T) = (0 -37.17739514656634) / Math.Sqrt((0.02658870712542621 / (23)) + (0.08285781696370748 / (26))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (26) - 2, .025) and -0.3067647420958625 = (28.449952733603105 - 37.17739514656634) / 28.449952733603105 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int16.ToString(value: -32768) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 47.07972254612325 > 40.00510731571173. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -59.5006756795596 (T) = (0 -47.244864121287684) / Math.Sqrt((0.504565467401582 / (25)) + (0.09595390931478781 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.2407454068752816 = (38.0778069856169 - 47.244864121287684) / 38.0778069856169 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Int32.ToString(value%3a%202147483647).html>) 50.48 ns 57.84 ns 1.15 0.00 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Int32.ToString(value%3a%20-2147483648).html>) 58.44 ns 66.68 ns 1.14 0.01 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Int32.TryFormat(value%3a%204).html>) 19.89 ns 32.78 ns 1.65 0.05 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Int32.TryFormat(value%3a%20-2147483648).html>) 58.48 ns 67.05 ns 1.15 0.01 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Int32.TryFormat(value%3a%202147483647).html>) 52.61 ns 59.80 ns 1.14 0.00 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Int32.ToString(value%3a%2012345).html>) 28.45 ns 37.30 ns 1.31 0.01 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Int32.TryFormat(value%3a%2012345).html>) 32.01 ns 41.46 ns 1.30 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Int32.ToString(value: 2147483647) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 57.83627723343707 > 52.98727145017573. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -76.87399049603007 (T) = (0 -57.87410113348923) / Math.Sqrt((0.09661061686274698 / (24)) + (0.14076414769628415 / (26))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (26) - 2, .025) and -0.14817563371626088 = (50.40526852688043 - 57.87410113348923) / 50.40526852688043 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ToString(value: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 66.67582755320618 > 61.527576174951456. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -79.32572761167238 (T) = (0 -67.12927776128699) / Math.Sqrt((0.14348920389754935 / (25)) + (0.15293383512510617 / (26))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (26) - 2, .025) and -0.14598749396106636 = (58.57767045018698 - 67.12927776128699) / 58.57767045018698 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryFormat(value: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 32.782895467628 > 20.697642559901304. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -41.18237625750828 (T) = (0 -34.31734156936501) / Math.Sqrt((0.048571649053965577 / (24)) + (3.275082660733824 / (26))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (26) - 2, .025) and -0.7523021258568191 = (19.584146513880956 - 34.31734156936501) / 19.584146513880956 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryFormat(value: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 67.0507000393293 > 61.442869084339925. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -148.351948096878 (T) = (0 -66.85930994785582) / Math.Sqrt((0.04453944173484399 / (27)) + (0.034747196233375466 / (25))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (25) - 2, .025) and -0.1393801246199376 = (58.68042499877558 - 66.85930994785582) / 58.68042499877558 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryFormat(value: 2147483647) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 59.79900977906343 > 55.15022888516503. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -117.79722603748677 (T) = (0 -59.90637282126425) / Math.Sqrt((0.019502828655986293 / (25)) + (0.08382916154496121 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.13968017904537458 = (52.5641964497824 - 59.90637282126425) / 52.5641964497824 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ToString(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 37.30207291453295 > 29.92750645615623. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -38.057167327232555 (T) = (0 -37.326765094187046) / Math.Sqrt((1.1707793333316412 / (23)) + (0.033032480196297474 / (26))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (26) - 2, .025) and -0.30358675301356125 = (28.633894144672038 - 37.326765094187046) / 28.633894144672038 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryFormat(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 41.461618729654255 > 33.64174689056041. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -197.14535537495777 (T) = (0 -41.51115452615153) / Math.Sqrt((0.04938214464463047 / (25)) + (0.00828831181218119 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.29346735465747015 = (32.09292787844987 - 41.51115452615153) / 32.09292787844987 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ParseAbsoluteUri - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Uri.ParseAbsoluteUri.html>) 659.18 ns 721.31 ns 1.09 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Uri*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Uri.ParseAbsoluteUri ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 721.3061634679533 > 694.875155670837. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -70.99404974247865 (T) = (0 -723.4231864854596) / Math.Sqrt((11.609148497028844 / (23)) + (5.882317886308811 / (26))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (26) - 2, .025) and -0.09158915111317173 = (662.7247859212719 - 723.4231864854596) / 662.7247859212719 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Buffers.Text.Tests.Base64Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Base64DecodeDestinationTooSmall - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Buffers.Text.Tests.Base64Tests.Base64DecodeDestinationTooSmall(NumberOfBytes%3a%201000).html>) 693.92 ns 799.44 ns 1.15 0.38 False
[Base64Decode - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Buffers.Text.Tests.Base64Tests.Base64Decode(NumberOfBytes%3a%201000).html>) 693.49 ns 747.76 ns 1.08 0.40 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Buffers.Text.Tests.Base64Tests.Base64DecodeDestinationTooSmall(NumberOfBytes: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 799.4399040186581 > 728.5801823234686. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -6.4658814480658275 (T) = (0 -755.8256119386037) / Math.Sqrt((769.3429308315053 / (26)) + (348.45996021660426 / (27))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (27) - 2, .025) and -0.05906107622430793 = (713.6751873019651 - 755.8256119386037) / 713.6751873019651 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Base64Tests.Base64Decode(NumberOfBytes: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 747.7580806459895 > 728.413624192859. IsChangePoint: Marked as a change because one of 7/15/2022 2:39:44 AM, 8/5/2022 10:47:59 PM, 10/3/2022 5:29:39 AM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -6.513602427399194 (T) = (0 -751.4288248735678) / Math.Sqrt((626.7070617715771 / (19)) + (343.7220425049511 / (32))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (32) - 2, .025) and -0.06080485450839494 = (708.3572644676478 - 751.4288248735678) / 708.3572644676478 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_Byte.ToString(value%3a%20255).html>) 20.90 ns 31.57 ns 1.51 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Byte*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Byte.ToString(value: 255) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 31.570430883505594 > 21.042714933498225. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -181.34167006030694 (T) = (0 -31.767383649391768) / Math.Sqrt((0.06423876744305015 / (24)) + (0.037783908440007286 / (25))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (25) - 2, .025) and -0.5858365266684936 = (20.03194094420836 - 31.767383649391768) / 20.03194094420836 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringD - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString%3a%20123).html>) 65.72 ns 77.86 ns 1.18 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 77.85821405558859 > 68.03874997892238. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -68.09163777540827 (T) = (0 -78.10221421488797) / Math.Sqrt((0.7702654404549802 / (24)) + (0.08220418017122641 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.19535363340836517 = (65.33816607240456 - 78.10221421488797) / 65.33816607240456 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_SByte.ToString(value%3a%20127).html>) 21.11 ns 32.39 ns 1.53 0.01 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_SByte.ToString(value%3a%20-128).html>) 30.07 ns 41.42 ns 1.38 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_SByte*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_SByte.ToString(value: 127) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 32.39072266974505 > 21.852139677294627. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -155.93319198995385 (T) = (0 -32.771752215809094) / Math.Sqrt((0.11296995381051365 / (25)) + (0.037198282804968376 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.5757288625075435 = (20.79783711244434 - 32.771752215809094) / 20.79783711244434 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_SByte.ToString(value: -128) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 41.41975235174107 > 31.898604366342983. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -67.74255044661754 (T) = (0 -41.78433025918779) / Math.Sqrt((0.5034915231800096 / (25)) + (0.14451291930072047 / (26))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (26) - 2, .025) and -0.3511580699862685 = (30.924827514527916 - 41.78433025918779) / 30.924827514527916 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Collections.IterateForEach(String).Dictionary(Size%3a%20512).html>) 3.92 μs 4.31 μs 1.10 0.18 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Collections.IterateForEach<String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 4.312351609377877 > 4.0950032455835. IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/18/2022 2:17:16 AM, 7/27/2022 9:27:06 AM, 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -16.919108993359014 (T) = (0 -4331.449599032735) / Math.Sqrt((10290.016772571467 / (24)) + (1512.3439126988055 / (26))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (26) - 2, .025) and -0.09432299000763279 = (3958.1089299810137 - 4331.449599032735) / 3958.1089299810137 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_UInt16.ToString(value%3a%2012345).html>) 27.72 ns 36.67 ns 1.32 0.01 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Tests.Perf_UInt16.ToString(value%3a%2065535).html>) 27.92 ns 37.12 ns 1.33 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_UInt16.ToString(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 36.67174324729929 > 29.138997462594876. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -140.04075336483737 (T) = (0 -36.71347980473281) / Math.Sqrt((0.023339275764534193 / (26)) + (0.08028812979138815 / (25))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (25) - 2, .025) and -0.3236544719565513 = (27.736452814959346 - 36.71347980473281) / 27.736452814959346 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt16.ToString(value: 65535) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 37.12193056965327 > 29.237171117707483. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -108.22217507063752 (T) = (0 -36.676359144337034) / Math.Sqrt((0.10013350871829466 / (27)) + (0.07417118477368966 / (25))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (25) - 2, .025) and -0.3176721803532682 = (27.83420617903923 - 36.676359144337034) / 27.83420617903923 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TimeSpan_ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString.html>) 242.67 ns 307.70 ns 1.27 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 307.6966870691704 > 254.54620971541297. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -64.20763362229576 (T) = (0 -296.4333458003417) / Math.Sqrt((3.8083183304419665 / (24)) + (13.445461340679222 / (25))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (25) - 2, .025) and -0.22065427218231956 = (242.84791570865514 - 296.4333458003417) / 242.84791570865514 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ChangeType - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=false/System.Perf_Convert.ChangeType.html>) 42.11 ns 48.86 ns 1.16 0.15 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Perf_Convert*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Perf_Convert.ChangeType ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 48.86115230348291 > 44.9667002699412. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/10/2022 9:53:50 PM falls between 10/2/2022 5:38:34 AM and 10/10/2022 9:53:50 PM. IsRegressionStdDev: Marked as regression because -26.471844836792272 (T) = (0 -49.538511793839504) / Math.Sqrt((0.8467973610856344 / (25)) + (0.6799302328940087 / (26))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (26) - 2, .025) and -0.15063978826798238 = (43.053014765296865 - 49.538511793839504) / 43.053014765296865 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)