search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 6/6/2022 2:10:15 PM #6094

Open performanceautofiler[bot] opened 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Replace_Char - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_String.Replace_Char(text%3a%20%22This%20is%20a%20very%20nice%20sentence%22%2c%20oldChar%3a%20%27z%27%2c%20newChar%3a%20%27y%27).html>) 9.99 ns 11.39 ns 1.14 0.12 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.38828034001448 > 10.604268294503541. IsChangePoint: Marked as a change because one of 4/21/2022 5:54:30 PM, 5/9/2022 11:56:12 AM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -27.907672521093755 (T) = (0 -11.841811231105874) / Math.Sqrt((0.017454147823955764 / (24)) + (0.15794312467688945 / (50))) is less than -1.9934635666612892 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (50) - 2, .025) and -0.1722149122018649 = (10.102082056661823 - 11.841811231105874) / 10.102082056661823 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_SByte.Parse(value%3a%20%22127%22).html>) 12.75 ns 14.21 ns 1.11 0.04 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_SByte.Parse(value%3a%20%22-128%22).html>) 13.09 ns 14.15 ns 1.08 0.03 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_SByte*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_SByte.Parse(value: "127") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.206435359624672 > 13.352595751888792. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 4/25/2022 5:27:54 PM, 5/2/2022 11:50:30 PM, 5/30/2022 12:57:05 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -21.710096509319364 (T) = (0 -14.475968439489629) / Math.Sqrt((0.0005881422908664534 / (24)) + (0.3269896483559066 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.13677221383575272 = (12.73427364189709 - 14.475968439489629) / 12.73427364189709 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_SByte.Parse(value: "-128") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.145269669337528 > 13.746387218044404. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 5/9/2022 5:23:46 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -18.417463648231408 (T) = (0 -14.414963917433573) / Math.Sqrt((0.015144752038063035 / (24)) + (0.22187437018071335 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.09911512011034444 = (13.115062884392309 - 14.414963917433573) / 13.115062884392309 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[HtmlDecode_Entities - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities.html>) 143.97 ns 160.10 ns 1.11 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 160.10335581556515 > 151.0790058484996. IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -23.617677620126596 (T) = (0 -159.00954456640162) / Math.Sqrt((1.9734521312770668 / (24)) + (15.314473362892464 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.10115424107471038 = (144.40260831326466 - 159.00954456640162) / 144.40260831326466 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Single.ToStringWithFormat(value%3a%20-3.4028235E%2b38%2c%20format%3a%20%22G17%22).html>) 164.98 ns 180.92 ns 1.10 0.07 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Single.TryParse(value%3a%20%2212345%22).html>) 61.05 ns 68.90 ns 1.13 0.04 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Single.Parse(value%3a%20%223.4028235E%2b38%22).html>) 94.99 ns 108.55 ns 1.14 0.09 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Single.TryParse(value%3a%20%22-3.4028235E%2b38%22).html>) 107.30 ns 115.78 ns 1.08 0.07 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Single.ToStringWithFormat(value%3a%203.4028235E%2b38%2c%20format%3a%20%22G17%22).html>) 162.84 ns 175.93 ns 1.08 0.05 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Single*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 180.91701097064293 > 173.0095474419317. IsChangePoint: Marked as a change because one of 5/26/2022 5:29:00 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -17.702641642303835 (T) = (0 -177.68198256670664) / Math.Sqrt((0.8319128192102379 / (24)) + (24.009236004857375 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.0762310174296663 = (165.09650780281333 - 177.68198256670664) / 165.09650780281333 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Single.TryParse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 68.90391021211465 > 64.21672216072652. IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/6/2022 7:24:37 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -25.77297461242468 (T) = (0 -67.32434187197765) / Math.Sqrt((0.2835566246608349 / (23)) + (2.1236096928469457 / (52))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (52) - 2, .025) and -0.0968162329124944 = (61.38160600814967 - 67.32434187197765) / 61.38160600814967 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Single.Parse(value: "3.4028235E+38") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 108.55470049540754 > 100.37758337586537. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -3.9592159878673674 (T) = (0 -107.81480468251047) / Math.Sqrt((142.8898823874732 / (28)) + (8.60047123018283 / (47))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (47) - 2, .025) and -0.0922171121659628 = (98.71188015787833 - 107.81480468251047) / 98.71188015787833 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Single.TryParse(value: "-3.4028235E+38") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 115.77935664681482 > 112.53666500823424. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 5/11/2022 7:02:26 PM, 5/15/2022 3:27:46 PM, 6/6/2022 10:12:27 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -11.091873143646467 (T) = (0 -113.40619846398633) / Math.Sqrt((1.4753808572108313 / (28)) + (10.872750770592804 / (47))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (47) - 2, .025) and -0.05499170515499934 = (107.49487214908926 - 113.40619846398633) / 107.49487214908926 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 175.93418349575194 > 170.97144237892115. IsChangePoint: Marked as a change because one of 5/9/2022 5:23:46 PM, 5/13/2022 5:31:28 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -17.626991678038696 (T) = (0 -175.46778554948105) / Math.Sqrt((0.5741376526199107 / (24)) + (23.92246771262071 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.07588687017586568 = (163.0912974342729 - 175.46778554948105) / 163.0912974342729 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.IO.Tests.StringReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%200%2c%20%20%20%200%5d).html>) 80.29 μs 84.68 μs 1.05 0.03 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%201%2c%20%20%20%201%5d).html>) 81.92 μs 87.83 μs 1.07 0.00 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StringReaderReadLineTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 0]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 84.68167567567568 > 84.27519759615383. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 9:09:01 PM, 5/20/2022 10:18:01 PM, 6/2/2022 5:58:32 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -46.09559406268896 (T) = (0 -84432.71632226881) / Math.Sqrt((3782.5173517165467 / (24)) + (409614.86589043716 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.051972751850932586 = (80261.31491876622 - 84432.71632226881) / 80261.31491876622 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StringReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 1]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 87.82518206304867 > 86.08225760798429. IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:37:06 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -26.02589423680721 (T) = (0 -88135.73443162536) / Math.Sqrt((1125082.6507766154 / (32)) + (426829.77580523235 / (43))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (43) - 2, .025) and -0.06689470508128369 = (82609.59025465455 - 88135.73443162536) / 82609.59025465455 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes%3a%20100).html>) 68.36 ns 82.10 ns 1.20 0.13 False
[WriteByteArrayAsBase64_NoEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes%3a%20100).html>) 68.98 ns 82.04 ns 1.19 0.12 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Base64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.10293851742564 > 71.82017506130556. IsChangePoint: Marked as a change because one of 4/25/2022 5:27:54 PM, 5/16/2022 6:32:29 PM, 5/24/2022 7:55:26 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -24.627416671640763 (T) = (0 -79.06112576698747) / Math.Sqrt((0.6499997652960561 / (24)) + (8.036236533425944 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.15454260664029826 = (68.47830934282639 - 79.06112576698747) / 68.47830934282639 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.04020919452724 > 71.26448088683752. IsChangePoint: Marked as a change because one of 4/25/2022 3:15:54 PM, 5/16/2022 6:32:29 PM, 5/24/2022 7:55:26 PM, 6/8/2022 9:45:10 AM, 6/9/2022 12:28:12 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -3.143610505167518 (T) = (0 -80.29177462882384) / Math.Sqrt((157.33198299273212 / (43)) + (4.544764007534288 / (32))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (32) - 2, .025) and -0.0826389839621672 = (74.16301816047448 - 80.29177462882384) / 74.16301816047448 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Decimal.ToString(value%3a%20123456.789).html>) 60.18 ns 65.42 ns 1.09 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Decimal*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Decimal.ToString(value: 123456.789) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 65.41605516101608 > 63.460000897386294. IsChangePoint: Marked as a change because one of 4/11/2022 1:47:52 PM, 4/14/2022 8:50:18 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -5.899344998272572 (T) = (0 -65.33667230592019) / Math.Sqrt((11.272132740834733 / (24)) + (0.4465804741118393 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.06661353411116923 = (61.256181565675114 - 65.33667230592019) / 61.256181565675114 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Xml.Linq.Perf_XName

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[NonEmptyNameSpaceToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Xml.Linq.Perf_XName.NonEmptyNameSpaceToString.html>) 20.41 ns 21.94 ns 1.07 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Linq.Perf_XName*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Xml.Linq.Perf_XName.NonEmptyNameSpaceToString ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.938727140953745 > 21.613417605660814. IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/14/2022 8:50:18 PM, 4/25/2022 11:05:49 AM, 5/9/2022 6:32:22 AM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -20.80754962283785 (T) = (0 -21.585546649073077) / Math.Sqrt((0.02202793058449577 / (24)) + (0.09275994772353384 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.053105516574622695 = (20.497040713719908 - 21.585546649073077) / 20.497040713719908 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value%3a%2018446744073709551615).html>) 19.74 ns 25.52 ns 1.29 0.08 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.521390474297036 > 20.620001580845447. IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -18.853010177085558 (T) = (0 -24.566436102760754) / Math.Sqrt((0.05446550303176786 / (24)) + (3.4177852998414897 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.2531373170537859 = (19.603945847306004 - 24.566436102760754) / 19.603945847306004 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAnyTwoValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Char).IndexOfAnyTwoValues(Size%3a%20512).html>) 13.22 ns 15.63 ns 1.18 0.05 False
[SequenceCompareTo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Char).SequenceCompareTo(Size%3a%20512).html>) 21.78 ns 24.25 ns 1.11 0.08 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Char>.IndexOfAnyTwoValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.629965237329191 > 13.88614601501657. IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -30.75202057942596 (T) = (0 -15.60571122149806) / Math.Sqrt((0.0001886334617559234 / (24)) + (0.3065339684770375 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.18045921558284267 = (13.22003421676272 - 15.60571122149806) / 13.22003421676272 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.SequenceCompareTo(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.252900031517616 > 22.79841623686469. IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 5/20/2022 10:18:01 PM, 6/8/2022 9:59:38 PM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -12.45112475002984 (T) = (0 -23.971424849359884) / Math.Sqrt((0.8814435029625298 / (42)) + (0.23484240265552714 / (33))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (33) - 2, .025) and -0.0953800017391956 = (21.884117668114374 - 23.971424849359884) / 21.884117668114374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Security.Cryptography.Tests.Perf_RandomNumberGenerator

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Security.Cryptography.Tests.Perf_RandomNumberGenerator.GetBytes.html>) 61.19 ns 67.00 ns 1.09 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Tests.Perf_RandomNumberGenerator*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Security.Cryptography.Tests.Perf_RandomNumberGenerator.GetBytes ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 66.99629722880758 > 64.22935151637111. IsChangePoint: Marked as a change because one of 4/22/2022 7:11:21 PM, 5/9/2022 11:56:12 AM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -24.22165046260898 (T) = (0 -66.46403627075388) / Math.Sqrt((0.12343510757633568 / (24)) + (2.392700880798503 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.09069140402879577 = (60.937526439879356 - 66.46403627075388) / 60.937526439879356 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Tests.Perf_Byte.Parse(value%3a%20%22255%22).html>) 12.85 ns 14.24 ns 1.11 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Byte*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Byte.Parse(value: "255") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.237586518694657 > 13.479090532042202. IsChangePoint: Marked as a change because one of 5/2/2022 11:50:30 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -19.214382921715288 (T) = (0 -14.546517471829219) / Math.Sqrt((0.018406746546972594 / (24)) + (0.35301019341285256 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.1309950995198431 = (12.861698055106386 - 14.546517471829219) / 12.861698055106386 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 1f1537378f722b125ac88ab1ed2fcbbeaeafa04b
Compare 54f104a16489b7a2fa5b9661e8030aa23aa494b0
Diff Diff

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.ContainsTrue(String).Queue(Size%3a%20512).html>) 411.24 μs 448.97 μs 1.09 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 448.9742023809524 > 431.9289860197367. IsChangePoint: Marked as a change because one of 4/16/2022 10:12:41 PM, 4/18/2022 2:31:51 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/25/2022 1:42:32 AM, 5/31/2022 6:06:22 PM, 6/6/2022 10:51:35 AM, 6/14/2022 8:14:23 AM falls between 6/5/2022 6:02:14 PM and 6/14/2022 8:14:23 AM. IsRegressionStdDev: Marked as regression because -63.37100093339434 (T) = (0 -450310.09831428766) / Math.Sqrt((791786.9414443945 / (24)) + (17642065.728646833 / (51))) is less than -1.99299712588925 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (51) - 2, .025) and -0.09484198580053403 = (411301.4518574814 - 450310.09831428766) / 411301.4518574814 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)