DrewScoggins / performance-2

This repo contains benchmarks used for testing the performance of all .NET Runtimes
MIT License
0 stars 0 forks source link

[Perf] Linux/x64: 48 Regressions on 10/3/2022 11:31:18 PM #8373

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in Benchstone.BenchI.CSieve

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/Benchstone.BenchI.CSieve.Test.html>) 7.09 ms 8.35 ms 1.18 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.CSieve*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Benchstone.BenchI.CSieve.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 8.350058191105768 > 7.467822278437502. IsChangePoint: Marked as a change because one of 8/31/2022 10:21:55 AM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -120.21860029203644 (T) = (0 -8348904.95666822) / Math.Sqrt((2714399339.3510094 / (26)) + (15639404.683202358 / (28))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (28) - 2, .025) and -0.1730480972550023 = (7117274.198905502 - 8348904.95666822) / 7117274.198905502 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in Layout.SearchLoops

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LoopGoto - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/Layout.SearchLoops.LoopGoto.html>) 71.13 ns 84.05 ns 1.18 0.01 False
[LoopReturn - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/Layout.SearchLoops.LoopReturn.html>) 101.06 ns 116.38 ns 1.15 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Layout.SearchLoops*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Layout.SearchLoops.LoopGoto ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 84.04694602848025 > 74.8812775301138. IsChangePoint: Marked as a change because one of 8/31/2022 10:21:55 AM, 9/2/2022 11:09:40 AM, 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -17.612029546739098 (T) = (0 -82.9877272574048) / Math.Sqrt((9.09168292519505 / (25)) + (0.06545221234486515 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.14732513554808307 = (72.3314818843929 - 82.9877272574048) / 72.3314818843929 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Layout.SearchLoops.LoopReturn ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 116.38268114566323 > 105.65477447873222. IsChangePoint: Marked as a change because one of 8/31/2022 10:21:55 AM, 9/2/2022 11:09:40 AM, 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -15.681416877238037 (T) = (0 -116.87801965195729) / Math.Sqrt((20.996781964782304 / (26)) + (2.2483471839295075 / (27))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (27) - 2, .025) and -0.14499615984586187 = (102.07721540978031 - 116.87801965195729) / 102.07721540978031 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryFormat2 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Version.TryFormat2.html>) 53.10 ns 79.33 ns 1.49 0.03 False
[ToString2 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Version.ToString2.html>) 81.34 ns 112.49 ns 1.38 0.02 False
[ToString3 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Version.ToString3.html>) 104.80 ns 150.11 ns 1.43 0.01 False
[ToStringL - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Version.ToStringL.html>) 311.19 ns 334.06 ns 1.07 0.00 False
[TryFormatL - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Version.TryFormatL.html>) 242.32 ns 266.02 ns 1.10 0.01 False
[TryFormat3 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Version.TryFormat3.html>) 75.36 ns 117.51 ns 1.56 0.01 False
[TryFormat4 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Version.TryFormat4.html>) 98.19 ns 153.86 ns 1.57 0.00 False
[ToString4 - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Version.ToString4.html>) 129.39 ns 188.36 ns 1.46 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Version.TryFormat2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 79.33138839438628 > 55.308291660221265. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -221.79316038155682 (T) = (0 -79.29652835480589) / Math.Sqrt((0.31007128654053323 / (25)) + (0.05997522144702315 / (28))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (28) - 2, .025) and -0.5090347765844236 = (52.547846865588525 - 79.29652835480589) / 52.547846865588525 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToString2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 112.49012757215438 > 85.65090861893906. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -85.75108658146407 (T) = (0 -113.71206401626613) / Math.Sqrt((0.5272528539793568 / (26)) + (3.3363397579358054 / (28))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (28) - 2, .025) and -0.3919629606511056 = (81.6918748779609 - 113.71206401626613) / 81.6918748779609 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToString3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 150.10554163145162 > 110.0457870913202. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -128.17390387681732 (T) = (0 -151.85975176175506) / Math.Sqrt((1.5504256557100424 / (25)) + (1.8882500094232855 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.4421589101525994 = (105.30029020566555 - 151.85975176175506) / 105.30029020566555 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToStringL ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 334.05672376785424 > 325.3443605757722. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -42.88810697606598 (T) = (0 -335.40557914189316) / Math.Sqrt((1.8329893471114107 / (22)) + (7.560686311985387 / (27))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (27) - 2, .025) and -0.08351406366401971 = (309.5535077852917 - 335.40557914189316) / 309.5535077852917 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.TryFormatL ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 266.0228005778543 > 254.43053138283196. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -36.55745850519723 (T) = (0 -267.08626903333914) / Math.Sqrt((8.018149178726807 / (26)) + (3.141191931528323 / (27))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (27) - 2, .025) and -0.09793995478452312 = (243.261271137324 - 267.08626903333914) / 243.261271137324 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.TryFormat3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 117.51431118454494 > 79.35789907628873. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -214.07908582808037 (T) = (0 -117.80053871043403) / Math.Sqrt((0.4469241663301918 / (25)) + (0.5870221896740992 / (28))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (28) - 2, .025) and -0.5580225450257197 = (75.60900776855536 - 117.80053871043403) / 75.60900776855536 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.TryFormat4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 153.86342118736113 > 103.27657291771422. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -191.40502607361213 (T) = (0 -155.1694633232303) / Math.Sqrt((0.7714990200891885 / (24)) + (1.4757303609458212 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.570903354288728 = (98.77721815260098 - 155.1694633232303) / 98.77721815260098 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToString4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 188.3614278022779 > 134.45725648998436. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -115.44405526136224 (T) = (0 -189.35042729175362) / Math.Sqrt((0.6007288160133457 / (25)) + (6.9301728422347955 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.4771465586993825 = (128.18662181935107 - 189.35042729175362) / 128.18662181935107 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[InstanceDelegate - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/PerfLabTests.LowLevelPerf.InstanceDelegate.html>) 174.04 μs 217.50 μs 1.25 0.03 False
[MeasureEvents - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/PerfLabTests.LowLevelPerf.MeasureEvents.html>) 15.48 ms 16.53 ms 1.07 0.27 False
[StaticIntPlus - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/PerfLabTests.LowLevelPerf.StaticIntPlus.html>) 154.67 μs 162.96 μs 1.05 0.03 False
[IntegerFormatting - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/PerfLabTests.LowLevelPerf.IntegerFormatting.html>) 4.91 ms 5.68 ms 1.16 0.02 False
[InterfaceInterfaceMethodSwitchCallType - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodSwitchCallType.html>) 543.54 μs 609.06 μs 1.12 0.02 False
[InterfaceInterfaceMethodLongHierarchy - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodLongHierarchy.html>) 282.86 μs 366.72 μs 1.30 0.03 False
[ObjectStringIsString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/PerfLabTests.LowLevelPerf.ObjectStringIsString.html>) 118.12 μs 152.25 μs 1.29 0.01 False
[InterfaceInterfaceMethod - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/PerfLabTests.LowLevelPerf.InterfaceInterfaceMethod.html>) 2.52 ms 2.98 ms 1.18 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### PerfLabTests.LowLevelPerf.InstanceDelegate ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 217.49833478009256 > 182.75511010817308. IsChangePoint: Marked as a change because one of 8/30/2022 6:17:03 PM, 9/2/2022 11:09:40 AM, 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -13.453838267697721 (T) = (0 -217732.42800913737) / Math.Sqrt((205694461.93195856 / (26)) + (348763.184876927 / (28))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (28) - 2, .025) and -0.21055986765656567 = (179860.93362786729 - 217732.42800913737) / 179860.93362786729 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.MeasureEvents ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 16.52828460989011 > 16.25617831875. IsChangePoint: Marked as a change because one of 8/30/2022 6:17:03 PM, 9/2/2022 11:09:40 AM, 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -5.403660325553844 (T) = (0 -16905065.94420651) / Math.Sqrt((659930159627.2211 / (25)) + (568494505240.2651 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.07484204302236633 = (15727953.752786662 - 16905065.94420651) / 15727953.752786662 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.StaticIntPlus ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 162.95654392361112 > 162.48868230143418. IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 10/5/2022 3:28:28 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -27.7207437460309 (T) = (0 -163039.62641277135) / Math.Sqrt((4603032.0044683665 / (38)) + (37510.21455357719 / (15))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (15) - 2, .025) and -0.06358447336352174 = (153292.59734035828 - 163039.62641277135) / 153292.59734035828 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.IntegerFormatting ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 5.683310342361111 > 5.171511079029448. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -5.626623918525108 (T) = (0 -5680810.295141149) / Math.Sqrt((283626193348.99774 / (24)) + (592103875.7119873 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.12079030964862211 = (5068575.491986663 - 5680810.295141149) / 5068575.491986663 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodSwitchCallType ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 609.0587668269231 > 570.9491469174818. IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/30/2022 9:25:07 PM, 9/2/2022 11:09:40 AM, 9/21/2022 9:54:19 PM, 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -8.525006400973526 (T) = (0 -624669.4781914047) / Math.Sqrt((1253631392.5772297 / (26)) + (448815738.9389132 / (28))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (28) - 2, .025) and -0.12282241111419932 = (556338.62666807 - 624669.4781914047) / 556338.62666807 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodLongHierarchy ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 366.720389484127 > 296.91127596819194. IsChangePoint: Marked as a change because one of 8/30/2022 6:17:03 PM, 8/31/2022 10:21:55 AM, 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -14.802155568551427 (T) = (0 -365654.6010006881) / Math.Sqrt((608711630.6662147 / (25)) + (16566674.210660499 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.25352954990637855 = (291700.0249639089 - 365654.6010006881) / 291700.0249639089 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.ObjectStringIsString ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 152.2527384329779 > 124.02729592330657. IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 8/31/2022 10:21:55 AM, 9/2/2022 11:09:40 AM, 10/5/2022 3:28:28 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -85.3388185289313 (T) = (0 -152320.32137281654) / Math.Sqrt((5621344.104374967 / (37)) + (13133.462006137976 / (16))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (16) - 2, .025) and -0.28035512254379324 = (118967.24486108859 - 152320.32137281654) / 118967.24486108859 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethod ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 2.9788192940705125 > 2.6493670227455355. IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -13.626684903117608 (T) = (0 -2978491.0053968546) / Math.Sqrt((21477681708.15127 / (25)) + (19985455.059305727 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.15494020639448436 = (2578913.5999474535 - 2978491.0053968546) / 2578913.5999474535 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_UInt32.ToString(value%3a%2012345).html>) 34.47 ns 38.91 ns 1.13 0.12 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_UInt32.TryFormat(value%3a%204294967295).html>) 54.33 ns 59.84 ns 1.10 0.02 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_UInt32.ToString(value%3a%204294967295).html>) 56.13 ns 59.99 ns 1.07 0.05 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_UInt32.TryFormat(value%3a%2012345).html>) 34.13 ns 41.25 ns 1.21 0.07 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_UInt32.TryFormat(value%3a%200).html>) 15.64 ns 30.00 ns 1.92 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_UInt32.ToString(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 38.90882543673501 > 36.170287029765866. IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -14.271007752568327 (T) = (0 -39.327108673912754) / Math.Sqrt((3.011048113348036 / (24)) + (0.630624229039883 / (26))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (26) - 2, .025) and -0.16334418259283287 = (33.80522227416951 - 39.327108673912754) / 33.80522227416951 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 59.84038812587762 > 57.13947872228367. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -34.634836675748105 (T) = (0 -59.03327239280758) / Math.Sqrt((0.27527273880200415 / (23)) + (0.17924185392635833 / (27))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (27) - 2, .025) and -0.0869922423852378 = (54.30882585074215 - 59.03327239280758) / 54.30882585074215 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.ToString(value: 4294967295) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 59.99321944118031 > 58.9263552131232. IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -13.667257021681964 (T) = (0 -60.19742248183035) / Math.Sqrt((2.5582966526460775 / (24)) + (0.29346812189271054 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.0843798882801091 = (55.51322293270031 - 60.19742248183035) / 55.51322293270031 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.TryFormat(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 41.2498676214093 > 35.87175421617204. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -40.229342712010364 (T) = (0 -41.439811311984855) / Math.Sqrt((0.7601165642897866 / (24)) + (0.1023059044401138 / (26))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (26) - 2, .025) and -0.22426685024324988 = (33.848675477696034 - 41.439811311984855) / 33.848675477696034 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt32.TryFormat(value: 0) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 29.999331564228942 > 16.515602572698423. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -49.852082072564 (T) = (0 -31.11570236216037) / Math.Sqrt((0.20926387713850678 / (24)) + (2.2901306440459575 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.9608013684020121 = (15.868870179093475 - 31.11570236216037) / 15.868870179093475 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_UInt16.ToString(value%3a%2012345).html>) 34.13 ns 39.00 ns 1.14 0.12 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_UInt16.ToString(value%3a%2065535).html>) 33.59 ns 39.28 ns 1.17 0.11 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_UInt16.ToString(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 39.00445108198787 > 35.79498199401506. IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -23.92520470740732 (T) = (0 -39.21643770871031) / Math.Sqrt((0.9879479936546858 / (24)) + (0.27760333466866 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.16060034124697378 = (33.78978647083227 - 39.21643770871031) / 33.78978647083227 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt16.ToString(value: 65535) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 39.2750646581382 > 35.30849722308636. IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -19.85654162675331 (T) = (0 -39.254677259559784) / Math.Sqrt((1.962897294209255 / (26)) + (0.5325448950293065 / (27))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (27) - 2, .025) and -0.18496101585815825 = (33.127399749207136 - 39.254677259559784) / 33.127399749207136 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Int32.TryFormat(value%3a%20-2147483648).html>) 63.44 ns 71.24 ns 1.12 0.01 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Int32.ToString(value%3a%20-2147483648).html>) 67.27 ns 72.55 ns 1.08 0.04 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Int32.TryFormat(value%3a%202147483647).html>) 53.82 ns 60.22 ns 1.12 0.02 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Int32.TryFormat(value%3a%204).html>) 16.03 ns 33.44 ns 2.09 0.12 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Int32.ToString(value%3a%2012345).html>) 36.93 ns 40.84 ns 1.11 0.11 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Int32.TryFormat(value%3a%2012345).html>) 33.79 ns 42.41 ns 1.26 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Int32.TryFormat(value: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 71.2361136350525 > 66.52607378229378. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -21.85900454150861 (T) = (0 -69.24712199800051) / Math.Sqrt((1.3476339181609092 / (26)) + (0.6720689000578559 / (27))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (27) - 2, .025) and -0.0958141860819555 = (63.192394182804954 - 69.24712199800051) / 63.192394182804954 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ToString(value: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 72.5491184848876 > 70.53507314574597. IsChangePoint: Marked as a change because one of 9/21/2022 9:54:19 PM, 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -12.002892355759238 (T) = (0 -72.78971819808469) / Math.Sqrt((6.204869936248024 / (23)) + (0.622201799816154 / (27))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (27) - 2, .025) and -0.097973639459712 = (66.29459540932409 - 72.78971819808469) / 66.29459540932409 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryFormat(value: 2147483647) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 60.22094770705023 > 56.514708731135066. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -26.51224297489006 (T) = (0 -59.87973523143452) / Math.Sqrt((0.9070937212836924 / (24)) + (0.28855814947676195 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.1080212039443073 = (54.04204812893118 - 59.87973523143452) / 54.04204812893118 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryFormat(value: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 33.44192383938467 > 16.84147053158671. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -181.22923850886497 (T) = (0 -33.361836078532164) / Math.Sqrt((0.11549710423645966 / (25)) + (0.11243672348712547 / (26))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (26) - 2, .025) and -1.0565625273387071 = (16.222135546592895 - 33.361836078532164) / 16.222135546592895 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ToString(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 40.84324131465418 > 38.68865647600488. IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -12.84656709985125 (T) = (0 -41.38802719333007) / Math.Sqrt((2.9291959382401513 / (24)) + (1.0105954849516972 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.14149404605886368 = (36.2577687866414 - 41.38802719333007) / 36.2577687866414 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryFormat(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 42.40553856379772 > 34.92722912781649. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -77.8257481126772 (T) = (0 -42.37652458579108) / Math.Sqrt((0.07124735080589402 / (24)) + (0.272738588913797 / (26))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (26) - 2, .025) and -0.2707415451670954 = (33.34787057758374 - 42.37652458579108) / 33.34787057758374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Numerics.Tests.Perf_BitOperations

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LeadingZeroCount_ulong - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_ulong.html>) 744.90 ns 1.10 μs 1.48 0.03 False
[LeadingZeroCount_uint - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_uint.html>) 668.03 ns 1.10 μs 1.64 0.03 False
[Log2_ulong - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Numerics.Tests.Perf_BitOperations.Log2_ulong.html>) 894.03 ns 1.32 μs 1.48 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BitOperations*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_ulong ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.103134802450392 > 782.1978117806847. IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -1036.320778168507 (T) = (0 -1103.3110857230588) / Math.Sqrt((2.062894119093593 / (24)) + (0.9287476680958434 / (28))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (28) - 2, .025) and -0.47969814915773445 = (745.63253752198 - 1103.3110857230588) / 745.63253752198 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BitOperations.LeadingZeroCount_uint ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.0981071343752826 > 701.5004525439659. IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -1966.1814324208126 (T) = (0 -1098.3022185241828) / Math.Sqrt((0.7454146771813229 / (26)) + (0.5374758729880522 / (28))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (28) - 2, .025) and -0.6438253628936942 = (668.1380171618691 - 1098.3022185241828) / 668.1380171618691 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BitOperations.Log2_ulong ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 1.3196406109936574 > 938.7702980902519. IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 8/22/2022 5:37:03 PM, 8/30/2022 12:00:50 PM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -1338.7639886021489 (T) = (0 -1320.615288952208) / Math.Sqrt((1.3722480937354582 / (25)) + (1.2489944587264858 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.47581862369238165 = (894.835766232664 - 1320.615288952208) / 894.835766232664 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in Benchstone.BenchI.BubbleSort

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/Benchstone.BenchI.BubbleSort.Test.html>) 16.07 μs 20.01 μs 1.24 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.BubbleSort*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Benchstone.BenchI.BubbleSort.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 20.00576061621967 > 16.87944682835783. IsChangePoint: Marked as a change because one of 8/31/2022 10:21:55 AM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -426.4728962956323 (T) = (0 -19995.27305766853) / Math.Sqrt((1252.979431684988 / (25)) + (909.299179414749 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.24261297306744595 = (16091.312010294969 - 19995.27305766853) / 16091.312010294969 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TimeSpan_ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString.html>) 233.32 ns 293.31 ns 1.26 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Xml.Tests.Perf_XmlConvert.TimeSpan_ToString ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 293.3099568059698 > 243.94541614429573. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -123.48818347179052 (T) = (0 -290.87341097489974) / Math.Sqrt((2.1337183448544974 / (24)) + (3.7704043460922714 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.25464298271241004 = (231.83759442551627 - 290.87341097489974) / 231.83759442551627 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_SByte.ToString(value%3a%20-128).html>) 39.67 ns 45.97 ns 1.16 0.10 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_SByte.ToString(value%3a%20127).html>) 29.94 ns 34.40 ns 1.15 0.17 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_SByte*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_SByte.ToString(value: -128) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 45.965687482153385 > 41.65070428088011. IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -16.128293051924082 (T) = (0 -46.06279035119912) / Math.Sqrt((3.0371853991454083 / (26)) + (1.9410184325238178 / (28))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (28) - 2, .025) and -0.1779419288739494 = (39.10446620677874 - 46.06279035119912) / 39.10446620677874 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_SByte.ToString(value: 127) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 34.400413857828696 > 31.62969522903811. IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -14.248964086514466 (T) = (0 -34.96632635975551) / Math.Sqrt((2.9052540852049673 / (25)) + (0.4426393407355862 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.17424987207467194 = (29.77758583697079 - 34.96632635975551) / 29.77758583697079 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ChangeType - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Perf_Convert.ChangeType.html>) 50.09 ns 58.17 ns 1.16 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Perf_Convert*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Perf_Convert.ChangeType ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 58.172406030969064 > 52.85907264834433. IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -16.00762098886106 (T) = (0 -57.10560833721096) / Math.Sqrt((1.1932720150108913 / (23)) + (3.2923300522057017 / (26))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (26) - 2, .025) and -0.13434581247865518 = (50.342327453415365 - 57.10560833721096) / 50.342327453415365 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in Benchstone.BenchF.SqMtx

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/Benchstone.BenchF.SqMtx.Test.html>) 623.60 ms 764.09 ms 1.23 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.SqMtx*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Benchstone.BenchF.SqMtx.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 764.0907556000001 > 654.9001596341348. IsChangePoint: Marked as a change because one of 7/28/2022 6:38:38 AM, 8/30/2022 6:17:03 PM, 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -16.653139973146622 (T) = (0 -764440428.5641569) / Math.Sqrt((1381812810477139 / (24)) + (716049633827.1786 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.1980891048086654 = (638049728.935844 - 764440428.5641569) / 638049728.935844 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in Benchstone.BenchI.BenchE

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/Benchstone.BenchI.BenchE.Test.html>) 760.75 ms 923.11 ms 1.21 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.BenchE*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Benchstone.BenchI.BenchE.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 923.10832725 > 799.147221565. IsChangePoint: Marked as a change because one of 8/31/2022 10:21:55 AM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -303.3900598254361 (T) = (0 -925890379.8059492) / Math.Sqrt((1309143957272.3699 / (25)) + (6533034726179.205 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.21620505920575542 = (761294629.386432 - 925890379.8059492) / 761294629.386432 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringD - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString%3a%20123).html>) 71.47 ns 83.43 ns 1.17 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 123) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 83.4265675858604 > 75.47898601887252. IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -32.159047856247795 (T) = (0 -83.6731534110157) / Math.Sqrt((2.691865965180592 / (25)) + (0.9074242240589887 / (28))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (28) - 2, .025) and -0.16801932822046625 = (71.63678835563088 - 83.6731534110157) / 71.63678835563088 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Int16.ToString(value%3a%2032767).html>) 36.01 ns 40.62 ns 1.13 0.13 False
[ToString - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Int16.ToString(value%3a%20-32768).html>) 47.27 ns 52.86 ns 1.12 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int16*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Int16.ToString(value: 32767) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 40.62480137369029 > 37.77127487704368. IsChangePoint: Marked as a change because one of 8/25/2022 6:55:57 AM, 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -13.522408509567097 (T) = (0 -40.796131519379045) / Math.Sqrt((3.1078096735520484 / (24)) + (0.7501060509047014 / (26))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (26) - 2, .025) and -0.15193648713334929 = (35.4152611494252 - 40.796131519379045) / 35.4152611494252 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int16.ToString(value: -32768) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 52.86360338273185 > 49.019578106519965. IsChangePoint: Marked as a change because one of 8/31/2022 10:21:55 AM, 9/29/2022 8:01:59 PM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -19.26580188540093 (T) = (0 -52.48908078985995) / Math.Sqrt((1.7524615843323976 / (24)) + (0.8031670954748431 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.1333549487975299 = (46.31301151113335 - 52.48908078985995) / 46.31301151113335 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in Benchstone.MDBenchI.MDNDhrystone

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/Benchstone.MDBenchI.MDNDhrystone.Test.html>) 663.16 ms 714.57 ms 1.08 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDNDhrystone*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Benchstone.MDBenchI.MDNDhrystone.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 714.5735297333333 > 693.1010994250001. IsChangePoint: Marked as a change because one of 7/27/2022 9:27:06 AM, 8/31/2022 10:21:55 AM, 9/2/2022 11:09:40 AM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -24.724071000108882 (T) = (0 -703282293.5818818) / Math.Sqrt((63233592776636.48 / (25)) + (29355058401494.348 / (27))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (27) - 2, .025) and -0.07164565006985942 = (656263843.8704394 - 703282293.5818818) / 656263843.8704394 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ParseAbsoluteUri - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Uri.ParseAbsoluteUri.html>) 665.10 ns 714.90 ns 1.07 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Uri*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Uri.ParseAbsoluteUri ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 714.9001007848187 > 698.3446582362538. IsChangePoint: Marked as a change because one of 9/2/2022 11:09:40 AM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -12.213561130002901 (T) = (0 -725.8385828032195) / Math.Sqrt((125.98803170225285 / (24)) + (448.8841577311754 / (27))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (27) - 2, .025) and -0.08542276697148012 = (668.7150895392 - 725.8385828032195) / 668.7150895392 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 23697bbc6b365fac14d986af7b23b2a908f3f6e0
Compare a6d396046ab327281e5a436b8d525f0b51d2ce0c
Diff Diff

Regressions in Benchstone.BenchI.IniArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/Benchstone.BenchI.IniArray.Test.html>) 187.19 ms 313.15 ms 1.67 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.IniArray*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Benchstone.BenchI.IniArray.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 313.146006 > 194.3689470690625. IsChangePoint: Marked as a change because one of 8/31/2022 10:21:55 AM, 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM falls between 10/2/2022 5:38:34 AM and 10/11/2022 2:16:14 AM. IsRegressionStdDev: Marked as regression because -271.8416725640976 (T) = (0 -313174657.46894306) / Math.Sqrt((5513848129260.673 / (25)) + (41930939741.99875 / (28))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (28) - 2, .025) and -0.6921361731157887 = (185076510.06141177 - 313174657.46894306) / 185076510.06141177 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)