search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 34 Regressions on 3/15/2023 12:03:45 AM #14266

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Aggregate - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.Aggregate(input%3a%20IEnumerable).html>) 605.38 ns 661.97 ns 1.09 0.26 False
[SingleWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input%3a%20IEnumerable).html>) 625.77 ns 667.85 ns 1.07 0.38 False
[WhereAny_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input%3a%20IEnumerable).html>) 648.12 ns 699.93 ns 1.08 0.08 False
[Where - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.Where(input%3a%20IEnumerable).html>) 1.05 μs 1.12 μs 1.06 0.04 False
[SelectToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20Array).html>) 246.73 ns 272.36 ns 1.10 0.06 False
[SelectToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20IList).html>) 412.04 ns 471.89 ns 1.15 0.08 False
[SkipHalfTakeHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.SkipHalfTakeHalf(input%3a%20IEnumerable).html>) 608.78 ns 658.92 ns 1.08 0.15 False
[SequenceEqual - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1%3a%20IEnumerable%2c%20input2%3a%20IEnumerable).html>) 1.01 μs 1.14 μs 1.12 0.18 False
[Prepend - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.Prepend(input%3a%20IEnumerable).html>) 2.78 μs 3.00 μs 1.08 0.30 False
[WhereSingle_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input%3a%20IEnumerable).html>) 644.30 ns 706.84 ns 1.10 0.05 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.Aggregate(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 661.9730249745159 > 648.6136214535748. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -4.823461527034242 (T) = (0 -646.9283231189411) / Math.Sqrt((364.9586262226801 / (28)) + (890.5559305824171 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.0506219626829741 = (615.757471380933 - 646.9283231189411) / 615.757471380933 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 667.8476380943869 > 654.4914664553046. IsChangePoint: Marked as a change because one of 2/24/2023 3:17:51 AM, 3/5/2023 11:36:44 PM, 3/17/2023 8:17:18 AM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -6.680884189266411 (T) = (0 -671.5112192336954) / Math.Sqrt((484.7604360048795 / (44)) + (505.09159355659733 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07124860877502218 = (626.8490934159267 - 671.5112192336954) / 626.8490934159267 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 699.9252237664733 > 680.7201591431524. IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 1/18/2023 3:20:52 PM, 2/6/2023 2:52:08 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -8.235962764081703 (T) = (0 -687.3153444432135) / Math.Sqrt((85.22638779627884 / (28)) + (479.6430282438574 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.054365703591836824 = (651.8756652476294 - 687.3153444432135) / 651.8756652476294 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Where(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1154126854828057 > 1.1055232586660024. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -10.140689515210513 (T) = (0 -1112.0380692742883) / Math.Sqrt((50.71248847214583 / (28)) + (888.1643979727659 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.05299702703483714 = (1056.0695241521303 - 1112.0380692742883) / 1056.0695241521303 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 272.3577231628281 > 258.91691149093646. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/9/2023 12:40:40 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -11.934709419610712 (T) = (0 -267.18991659016916) / Math.Sqrt((6.8389605603208885 / (28)) + (80.41575817165926 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.08137635009937351 = (247.0831885361795 - 267.18991659016916) / 247.0831885361795 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 471.8864461876564 > 434.675468770804. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/9/2023 12:40:40 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.363561694210494 (T) = (0 -455.86107877238004) / Math.Sqrt((5.1580890266096 / (28)) + (624.6793486618997 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.10208125994682793 = (413.6365396453379 - 455.86107877238004) / 413.6365396453379 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SkipHalfTakeHalf(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 658.9151202013452 > 639.9453847097661. IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/14/2023 11:21:41 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -10.911211473027418 (T) = (0 -647.7946643687045) / Math.Sqrt((6.790736000568051 / (28)) + (379.80551659258964 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.0633068153240323 = (609.2264763405052 - 647.7946643687045) / 609.2264763405052 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: IEnumerable, input2: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1355220858383437 > 1.0643488233990366. IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 3/15/2023 5:33:41 AM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.56942330738282 (T) = (0 -1129.0007030691686) / Math.Sqrt((218.72326312158202 / (30)) + (3476.5168791171704 / (29))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (29) - 2, .025) and -0.10568583301888086 = (1021.086342389529 - 1129.0007030691686) / 1021.086342389529 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Prepend(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.996568394938713 > 2.907346916516622. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -6.141877654009774 (T) = (0 -2954.6632964827304) / Math.Sqrt((11618.527224189023 / (28)) + (16521.05855429418 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.06837454835560586 = (2765.5687801908193 - 2954.6632964827304) / 2765.5687801908193 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 706.8398895907203 > 679.2855225890617. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -8.974206463163782 (T) = (0 -696.7214298377355) / Math.Sqrt((49.75361673983378 / (28)) + (809.260690273098 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.07297772529110977 = (649.3344767699701 - 696.7214298377355) / 649.3344767699701 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.AddGivenSize(String).List(Size%3a%20512).html>) 1.73 μs 1.87 μs 1.08 0.04 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.AddGivenSize(String).Queue(Size%3a%20512).html>) 2.06 μs 2.39 μs 1.16 0.51 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<String>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.872252994842216 > 1.8216587727379534. IsChangePoint: Marked as a change because one of 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -25.08202160612491 (T) = (0 -1881.5519672929706) / Math.Sqrt((6.828176274329146 / (28)) + (1057.6204718289594 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.08476417875914405 = (1734.5262722864502 - 1881.5519672929706) / 1734.5262722864502 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.AddGivenSize<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3914202279202277 > 2.0252932722836876. IsChangePoint: Marked as a change because one of 1/19/2023 7:53:02 PM, 3/16/2023 8:14:39 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -5.625223718547157 (T) = (0 -2225.603726123397) / Math.Sqrt((13318.734721075209 / (41)) + (46275.58818655382 / (18))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (18) - 2, .025) and -0.15742033852865284 = (1922.9001357904688 - 2225.603726123397) / 1922.9001357904688 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Date_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options%3a%20Compiled).html>) 105.91 ns 128.01 ns 1.21 0.05 False
[Date_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 107.15 ns 131.47 ns 1.23 0.08 False
[Backtracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options%3a%20IgnoreCase%2c%20Compiled).html>) 124.96 ns 133.93 ns 1.07 0.07 False
[Date_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 64.81 ns 72.82 ns 1.12 0.09 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 128.00821981013425 > 111.78526046076479. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 6:24:13 PM, 1/25/2023 9:58:39 PM, 2/9/2023 12:40:40 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.068406866618318 (T) = (0 -124.29520153289056) / Math.Sqrt((1.1279753391053862 / (28)) + (117.00937892460136 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.16617922801092827 = (106.58327514964603 - 124.29520153289056) / 106.58327514964603 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 131.4749944752738 > 112.84219692764826. IsChangePoint: Marked as a change because one of 1/19/2023 12:32:21 PM, 2/9/2023 12:40:40 PM, 2/20/2023 10:31:48 AM, 3/3/2023 4:59:02 PM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -64.37085695456977 (T) = (0 -131.3428710548161) / Math.Sqrt((7.795496017561647 / (56)) + (0.013103267514300592 / (3))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (56) + (3) - 2, .025) and -0.22805335921426717 = (106.95208809074214 - 131.3428710548161) / 106.95208809074214 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 133.92761486592443 > 132.23717740934214. IsChangePoint: Marked as a change because one of 1/5/2023 11:35:29 PM, 2/24/2023 9:01:57 AM, 2/28/2023 10:21:25 PM, 3/13/2023 2:14:21 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -6.271597440473676 (T) = (0 -130.22200967699854) / Math.Sqrt((37.03258072751583 / (28)) + (5.712436548971013 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.06283424392897984 = (122.52334775703761 - 130.22200967699854) / 122.52334775703761 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 72.81554203047673 > 67.66683954528196. IsChangePoint: Marked as a change because one of 2/14/2023 5:47:38 PM, 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/13/2023 2:14:21 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.624848309251204 (T) = (0 -71.44358130869404) / Math.Sqrt((0.8203259698772776 / (19)) + (19.583199217221516 / (40))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (40) - 2, .025) and -0.10905554556588677 = (64.41839779290812 - 71.44358130869404) / 64.41839779290812 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetByteCount - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Tests.Perf_Encoding.GetByteCount(size%3a%20512%2c%20encName%3a%20%22utf-8%22).html>) 20.50 ns 25.70 ns 1.25 0.08 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_Encoding.GetByteCount(size: 512, encName: "utf-8") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.699042120357912 > 22.258380048700666. IsChangePoint: Marked as a change because one of 1/27/2023 7:12:54 PM, 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -8.783152430699818 (T) = (0 -25.01499295194046) / Math.Sqrt((0.1400822478120252 / (28)) + (5.9752817999795855 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.18502991381562067 = (21.10916581961707 - 25.01499295194046) / 21.10916581961707 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.IterateFor(Int32).ImmutableSortedSet(Size%3a%20512).html>) 5.84 μs 6.30 μs 1.08 0.12 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.297477608300447 > 6.121700024441238. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 11:43:56 PM, 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -7.670426689476263 (T) = (0 -6293.338080253745) / Math.Sqrt((5520.639457441548 / (28)) + (114191.65218226556 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.08216579646874868 = (5815.502671392634 - 6293.338080253745) / 5815.502671392634 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAnyFourValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Memory.Span(Byte).IndexOfAnyFourValues(Size%3a%20512).html>) 13.02 ns 14.14 ns 1.09 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.13893870309976 > 13.671752694086962. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -15.47011101404966 (T) = (0 -14.263515283195416) / Math.Sqrt((2.1966928448950787E-05 / (28)) + (0.1997747316575576 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.09537764543534943 = (13.021550460367932 - 14.263515283195416) / 13.021550460367932 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToStream(Mode%3a%20Reflection).html>) 7.17 μs 8.49 μs 1.18 0.28 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.488619471863828 > 7.55586662173765. IsChangePoint: Marked as a change because one of 1/17/2023 5:52:57 PM, 2/9/2023 4:27:10 AM, 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -5.985373779478639 (T) = (0 -8218.822994912243) / Math.Sqrt((154649.60405869913 / (28)) + (451155.28515356214 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.11506016580530455 = (7370.7439714488855 - 8218.822994912243) / 7370.7439714488855 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 17.96 ns 21.49 ns 1.20 0.05 False
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 144.06 ns 171.13 ns 1.19 0.05 False
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 143.78 ns 171.15 ns 1.19 0.04 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, Ordinal, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.488494325518868 > 19.108210503614043. IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 2/9/2023 4:27:10 AM, 2/16/2023 9:36:08 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.273518129073775 (T) = (0 -20.414874157804903) / Math.Sqrt((0.05929464740983003 / (28)) + (1.6703881323660095 / (30))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (30) - 2, .025) and -0.12259557924960562 = (18.185421834148983 - 20.414874157804903) / 18.185421834148983 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 171.13291770533894 > 151.33578296936622. IsChangePoint: Marked as a change because one of 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.166275268098993 (T) = (0 -164.39013947861739) / Math.Sqrt((0.33603991146155976 / (28)) + (148.66293236438167 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.13928807550781014 = (144.291986383992 - 164.39013947861739) / 144.291986383992 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 171.14519849626015 > 151.25798870552777. IsChangePoint: Marked as a change because one of 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.07238728099435 (T) = (0 -164.2931689960972) / Math.Sqrt((0.04016090233270838 / (28)) + (152.34757382573744 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.139515552219933 = (144.17808399019347 - 164.2931689960972) / 144.17808399019347 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Doubles

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteDoubles - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted%3a%20True%2c%20SkipValidation%3a%20True).html>) 21.49 ms 22.58 ms 1.05 0.04 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Doubles*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.58354181818182 > 22.4970185. IsChangePoint: Marked as a change because one of 1/17/2023 6:49:05 AM, 2/24/2023 8:06:06 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -12.698930446637643 (T) = (0 -22708463.221509673) / Math.Sqrt((128849480860.86531 / (28)) + (113879387919.49176 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.053597677107410494 = (21553258.625108592 - 22708463.221509673) / 21553258.625108592 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ICollection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.AddGivenSize(Int32).ICollection(Size%3a%20512).html>) 1.36 μs 1.64 μs 1.20 0.10 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<Int32>.ICollection(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6374127200971464 > 1.432446382098855. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.354100200578316 (T) = (0 -1571.883659385063) / Math.Sqrt((11.474561352960475 / (28)) + (15509.315228885194 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.15361587690120734 = (1362.5711043500792 - 1571.883659385063) / 1362.5711043500792 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20fr).html>) 170.07 ns 199.71 ns 1.17 0.05 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20ja).html>) 168.78 ns 196.38 ns 1.16 0.06 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20).html>) 167.93 ns 198.13 ns 1.18 0.03 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: fr) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 199.7137681631165 > 179.16255775533443. IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 1/18/2023 3:20:52 PM, 2/17/2023 5:12:50 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -16.81959310051656 (T) = (0 -194.1772977686033) / Math.Sqrt((2.561262436111943 / (28)) + (59.30864674262537 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.13978474584011466 = (170.36313082561807 - 194.1772977686033) / 170.36313082561807 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ja) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 196.37617235053065 > 176.5600535984116. IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 1/18/2023 3:20:52 PM, 2/21/2023 5:46:41 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -34.69142147443889 (T) = (0 -196.4724593342386) / Math.Sqrt((2.0417114256472924 / (28)) + (17.953860451400537 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.16629429850845617 = (168.45873257333264 - 196.4724593342386) / 168.45873257333264 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 198.13235713544907 > 175.9064489739677. IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -16.07495541074479 (T) = (0 -193.66214149587867) / Math.Sqrt((0.9439977193464116 / (28)) + (79.05800544164958 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.15397302534599513 = (167.8220697037637 - 193.66214149587867) / 167.8220697037637 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.IterateForEach(Int32).Queue(Size%3a%20512).html>) 1.59 μs 1.72 μs 1.08 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7246643467373695 > 1.6723418929177543. IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.06980870288975 (T) = (0 -1689.7640381783976) / Math.Sqrt((2.657748440220598 / (28)) + (3562.5540585637964 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.0610800490493964 = (1592.4944019937313 - 1689.7640381783976) / 1592.4944019937313 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FooObjIsFoo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/PerfLabTests.CastingPerf2.CastingPerf.FooObjIsFoo.html>) 556.86 μs 612.73 μs 1.10 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.CastingPerf2.CastingPerf.FooObjIsFoo ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 612.7295673076923 > 584.8925306919642. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.264551780946848 (T) = (0 -598352.0786332524) / Math.Sqrt((73184.96274360204 / (28)) + (615985049.8498245 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.07414159852119338 = (557051.3975597107 - 598352.0786332524) / 557051.3975597107 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ObjectStringIsString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/PerfLabTests.LowLevelPerf.ObjectStringIsString.html>) 69.64 μs 83.56 μs 1.20 0.03 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.ObjectStringIsString ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 83.56058217738546 > 73.10943805555556. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/9/2023 4:27:10 AM, 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.272952351585408 (T) = (0 -79957.54343137551) / Math.Sqrt((237.55735082719653 / (28)) + (38402555.04666665 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.14821121614848323 = (69636.61590032373 - 79957.54343137551) / 69636.61590032373 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EnumerateActivityTagsSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Diagnostics.Perf_Activity.EnumerateActivityTagsSmall.html>) 44.92 ns 54.06 ns 1.20 0.18 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Diagnostics.Perf_Activity.EnumerateActivityTagsSmall ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 54.06354479620618 > 48.05120874427849. IsChangePoint: Marked as a change because one of 1/18/2023 3:20:52 PM, 2/13/2023 6:54:14 PM, 2/23/2023 8:54:44 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.110178365817202 (T) = (0 -50.7766304041184) / Math.Sqrt((1.7592613323811164 / (28)) + (6.730830810387768 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.10488774774748 = (45.956370235470565 - 50.7766304041184) / 45.956370235470565 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Add_Remove_SteadyState<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentBag - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/System.Collections.Tests.Add_Remove_SteadyState(String).ConcurrentBag(Count%3a%20512).html>) 39.43 ns 41.45 ns 1.05 0.07 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Add_Remove_SteadyState<String>.ConcurrentBag(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 41.45427372157048 > 41.36091361449625. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -12.36542841623914 (T) = (0 -41.84232133164783) / Math.Sqrt((0.15758639627391013 / (28)) + (0.9640761775735779 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.060035540670132344 = (39.47256457570846 - 41.84232133164783) / 39.47256457570846 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline a78f9bde2edd384f5d26c9d394eb7fcc5f48a664
Compare 1cfda5e9a887d39004644c2f96a87b31fcca0ac7
Diff Diff
Configs CompliationMode:tiered, RunKind:micro

Regressions in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Ref - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_Windows 10.0.19042/amd/Span.IndexerBench.Ref(length%3a%201024).html>) 577.15 ns 638.51 ns 1.11 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.IndexerBench*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Span.IndexerBench.Ref(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 638.5089249076734 > 606.0767002314395. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/9/2023 12:40:40 PM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/21/2023 3:33:39 AM falls between 3/11/2023 9:11:51 PM and 3/21/2023 3:33:39 AM. IsRegressionStdDev: Marked as regression because -9.274459263976388 (T) = (0 -622.8304311763043) / Math.Sqrt((0.050640023434839025 / (28)) + (751.1487641219823 / (31))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (31) - 2, .025) and -0.079100456389488 = (577.1755794267789 - 622.8304311763043) / 577.1755794267789 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)