search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 25 Regressions on 1/9/2023 10:41:51 AM #11675

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5da4a9e919dcee35f831ab69b6e475baaf798875
Compare dcda1e08cb8f421877388be2d685c009a31b1845
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Aggregate - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.Aggregate(input%3a%20IEnumerable).html>) 455.41 ns 499.23 ns 1.10 0.07 False
[CastToSameType - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.CastToSameType(input%3a%20IEnumerable).html>) 371.02 ns 397.23 ns 1.07 0.06 False
[SelectToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.SelectToArray(input%3a%20Range).html>) 200.09 ns 220.72 ns 1.10 0.06 False
[Average - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.Average(input%3a%20IEnumerable).html>) 347.12 ns 367.94 ns 1.06 0.00 False
[WhereLast_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input%3a%20Array).html>) 193.27 ns 219.66 ns 1.14 0.06 False
[Where - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.Where(input%3a%20Array).html>) 535.19 ns 575.38 ns 1.08 0.06 False
[SingleWithPredicate_FirstElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input%3a%20List).html>) 721.07 ns 763.32 ns 1.06 0.03 False
[WhereAny_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input%3a%20Array).html>) 194.80 ns 216.02 ns 1.11 0.06 False
[WhereSingleOrDefault_LastElementMatches - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input%3a%20Array).html>) 193.01 ns 212.11 ns 1.10 0.05 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.Aggregate(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 499.23422593110064 > 478.4603591781991. IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -19.193075603757602 (T) = (0 -494.6924908659078) / Math.Sqrt((0.33027326812528085 / (17)) + (149.32185519541505 / (36))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (36) - 2, .025) and -0.0860141474314087 = (455.51201338944986 - 494.6924908659078) / 455.51201338944986 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 397.225219156743 > 389.9619598753325. IsChangePoint: Marked as a change because one of 10/31/2022 4:38:04 PM, 11/7/2022 4:10:19 PM, 11/10/2022 9:13:35 AM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -12.052643291535858 (T) = (0 -396.04737990781973) / Math.Sqrt((39.32440980561685 / (18)) + (56.518983444600835 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.06266375324609183 = (372.6930354950227 - 396.04737990781973) / 372.6930354950227 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Range) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 220.72278440488373 > 209.9835352521052. IsChangePoint: Marked as a change because one of 10/31/2022 4:38:04 PM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 1/3/2023 6:29:42 AM, 1/5/2023 12:24:13 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -11.838957955776403 (T) = (0 -219.44283070925667) / Math.Sqrt((25.398592570076318 / (18)) + (38.89284892538183 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.09308199112079366 = (200.75605717760527 - 219.44283070925667) / 200.75605717760527 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Average(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 367.9436598577977 > 364.72842857180177. IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -167.93659115513216 (T) = (0 -367.94645181503944) / Math.Sqrt((0.2489805927529808 / (27)) + (0.15446738000601334 / (27))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (27) - 2, .025) and -0.05908877204190219 = (347.4179516657947 - 367.94645181503944) / 347.4179516657947 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 219.65647268513294 > 203.1608079030798. IsChangePoint: Marked as a change because one of 10/31/2022 4:38:04 PM, 11/7/2022 4:10:19 PM, 11/18/2022 8:48:12 PM, 12/1/2022 7:29:59 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -12.228889241345877 (T) = (0 -214.77611830989764) / Math.Sqrt((27.788765222414053 / (18)) + (40.64591168636612 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.10264190658284583 = (194.78319935753382 - 214.77611830989764) / 194.78319935753382 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Where(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 575.3788234712704 > 562.1206560608915. IsChangePoint: Marked as a change because one of 10/31/2022 4:38:04 PM, 11/7/2022 4:10:19 PM, 11/10/2022 9:13:35 AM, 12/1/2022 7:29:59 PM, 12/3/2022 7:24:45 AM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -12.57792965808625 (T) = (0 -572.9398153714332) / Math.Sqrt((91.94237291606703 / (18)) + (89.7996154344989 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.06443045373645674 = (538.2595108588351 - 572.9398153714332) / 538.2595108588351 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: List) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 763.3204017012463 > 756.8929596578947. IsChangePoint: Marked as a change because one of 11/18/2022 8:48:12 PM, 12/1/2022 7:29:59 PM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 1/3/2023 6:29:42 AM, 1/5/2023 12:24:13 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -9.788085451787799 (T) = (0 -783.3882907147039) / Math.Sqrt((316.3021398061895 / (18)) + (600.0688323194424 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.07887992705815375 = (726.1125831220305 - 783.3882907147039) / 726.1125831220305 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 216.01751484928153 > 204.3094839182893. IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -11.596853594493114 (T) = (0 -214.0446459948715) / Math.Sqrt((25.373065729696858 / (18)) + (34.14540093355239 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.09074893549024518 = (196.2364014581115 - 214.0446459948715) / 196.2364014581115 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.WhereSingleOrDefault_LastElementMatches(input: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 212.10653571937078 > 202.8538290643974. IsChangePoint: Marked as a change because one of 10/31/2022 4:38:04 PM, 11/7/2022 4:10:19 PM, 11/18/2022 8:48:12 PM, 12/1/2022 7:29:59 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -10.817432773664287 (T) = (0 -210.47848169122554) / Math.Sqrt((24.061710575932395 / (18)) + (28.862008685485375 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.08126462111703509 = (194.65954733059147 - 210.47848169122554) / 194.65954733059147 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5da4a9e919dcee35f831ab69b6e475baaf798875
Compare dcda1e08cb8f421877388be2d685c009a31b1845
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexerCheckPathLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_String.IndexerCheckPathLength.html>) 82.77 ns 112.71 ns 1.36 0.07 False
[Insert - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_String.Insert(s1%3a%20%22dzsdzsDDZSDZSDZSddsz%22%2c%20i%3a%207%2c%20s2%3a%20%22Test%22).html>) 15.42 ns 17.21 ns 1.12 0.01 False
[Remove_IntInt - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_String.Remove_IntInt(s%3a%20%22dzsdzsDDZSDZSDZSddsz%22%2c%20i1%3a%2010%2c%20i2%3a%201).html>) 13.83 ns 14.95 ns 1.08 0.04 False
[Remove_IntInt - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_String.Remove_IntInt(s%3a%20%22dzsdzsDDZSDZSDZSddsz%22%2c%20i1%3a%207%2c%20i2%3a%204).html>) 13.38 ns 14.68 ns 1.10 0.03 False
[Insert - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_String.Insert(s1%3a%20%22Test%22%2c%20i%3a%202%2c%20s2%3a%20%22%20Test%22).html>) 14.56 ns 15.75 ns 1.08 0.02 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.IndexerCheckPathLength ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 112.71362095891703 > 86.91545599806743. IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -12.195476373328008 (T) = (0 -110.11425275651297) / Math.Sqrt((46.174267369583305 / (19)) + (71.7689138124794 / (36))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (36) - 2, .025) and -0.30368734701963723 = (84.46369676613448 - 110.11425275651297) / 84.46369676613448 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.206198012375673 > 16.182051735581428. IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 1/13/2023 4:18:46 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -36.742510926111244 (T) = (0 -17.06311851316192) / Math.Sqrt((0.04925660701861988 / (32)) + (0.008509059440428355 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.10436535263005318 = (15.45061013778275 - 17.06311851316192) / 15.45061013778275 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.94681389990589 > 14.525309522030922. IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 12/14/2022 5:20:21 PM, 1/13/2023 4:18:46 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -22.211289591215362 (T) = (0 -15.033928101746818) / Math.Sqrt((0.08893948685244389 / (33)) + (0.007746736775226047 / (22))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (22) - 2, .025) and -0.08879804833877586 = (13.807820582232585 - 15.033928101746818) / 13.807820582232585 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 7, i2: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.684601180984883 > 14.054411144151118. IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 12/14/2022 8:02:51 PM, 1/13/2023 4:18:46 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -31.720120814518268 (T) = (0 -14.742305613888051) / Math.Sqrt((0.04803450854629171 / (32)) + (0.008436187994094194 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.10302892664738236 = (13.365293745012446 - 14.742305613888051) / 13.365293745012446 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.754461926692374 > 15.302199155473708. IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 1/13/2023 4:18:46 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -34.17422582869911 (T) = (0 -15.593426741462565) / Math.Sqrt((0.018232632547926928 / (32)) + (0.008944027036010585 / (22))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (22) - 2, .025) and -0.0735121979173238 = (14.525616729567416 - 15.593426741462565) / 14.525616729567416 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5da4a9e919dcee35f831ab69b6e475baaf798875
Compare dcda1e08cb8f421877388be2d685c009a31b1845
Diff Diff

Regressions in PerfLabTests.DelegatePerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DelegateInvoke - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/PerfLabTests.DelegatePerf.DelegateInvoke.html>) 304.33 μs 347.89 μs 1.14 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.DelegatePerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.DelegatePerf.DelegateInvoke ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 347.8921467948718 > 319.8545673076923. IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 12/10/2022 11:25:25 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -11.886382572442379 (T) = (0 -344688.148969839) / Math.Sqrt((104834974.84289557 / (18)) + (150715407.4883835 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.12247022004983422 = (307079.99447373836 - 344688.148969839) / 307079.99447373836 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5da4a9e919dcee35f831ab69b6e475baaf798875
Compare dcda1e08cb8f421877388be2d685c009a31b1845
Diff Diff

Regressions in Benchstone.BenchI.XposMatrix

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Benchstone.BenchI.XposMatrix.Test.html>) 15.11 μs 17.13 μs 1.13 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.XposMatrix*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.XposMatrix.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.13410668174342 > 15.864025564903848. IsChangePoint: Marked as a change because one of 10/31/2022 4:38:04 PM, 11/18/2022 8:48:12 PM, 12/12/2022 5:25:39 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -11.789730972277713 (T) = (0 -16963.305893016124) / Math.Sqrt((239004.06949688258 / (18)) + (349745.9171051478 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.11780352463479761 = (15175.570231412787 - 16963.305893016124) / 15175.570231412787 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5da4a9e919dcee35f831ab69b6e475baaf798875
Compare dcda1e08cb8f421877388be2d685c009a31b1845
Diff Diff

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Int64.ParseSpan(value%3a%20%22-9223372036854775808%22).html>) 26.92 ns 30.03 ns 1.12 0.06 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Int64.ToString(value%3a%20-9223372036854775808).html>) 28.74 ns 31.84 ns 1.11 0.07 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 30.029392490105444 > 28.268482665662493. IsChangePoint: Marked as a change because one of 11/18/2022 8:48:12 PM, 12/13/2022 4:26:35 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -13.827081808993858 (T) = (0 -30.017871577882563) / Math.Sqrt((0.6544172141708096 / (18)) + (0.3239206406028884 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.10876784658256057 = (27.073180080396014 - 30.017871577882563) / 27.073180080396014 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int64.ToString(value: -9223372036854775808) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.835128874813186 > 30.061864584155988. IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -9.610159106568556 (T) = (0 -31.251558645668556) / Math.Sqrt((0.7054644008365252 / (18)) + (1.2176867105461913 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.09062468310395692 = (28.654732585664117 - 31.251558645668556) / 28.654732585664117 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5da4a9e919dcee35f831ab69b6e475baaf798875
Compare dcda1e08cb8f421877388be2d685c009a31b1845
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern%3a%20%22(%3fi)Sherlock%22%2c%20Options%3a%20Compiled).html>) 92.58 μs 100.10 μs 1.08 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 100.09975688125569 > 96.9699792949443. IsChangePoint: Marked as a change because one of 11/18/2022 8:48:12 PM, 1/3/2023 6:29:42 AM, 1/5/2023 12:24:13 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -9.976590818950893 (T) = (0 -98890.23312728132) / Math.Sqrt((3057976.093228697 / (18)) + (7095776.944046294 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.06509468940381835 = (92846.42399506719 - 98890.23312728132) / 92846.42399506719 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5da4a9e919dcee35f831ab69b6e475baaf798875
Compare dcda1e08cb8f421877388be2d685c009a31b1845
Diff Diff

Regressions in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IndexerSet(String).Span(Size%3a%20512).html>) 180.32 ns 235.83 ns 1.31 0.14 False
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IndexerSet(String).ConcurrentDictionary(Size%3a%20512).html>) 26.36 μs 29.40 μs 1.12 0.04 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 235.83405448227833 > 189.0924642799876. IsChangePoint: Marked as a change because one of 11/1/2022 2:33:16 AM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 1/5/2023 12:24:13 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -6.943918593535711 (T) = (0 -236.50133031832658) / Math.Sqrt((797.3919013703528 / (18)) + (49.43807660302037 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.24754646138960018 = (189.57316431717956 - 236.50133031832658) / 189.57316431717956 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IndexerSet<String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.40450337080682 > 27.767026037471698. IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -18.16861160248942 (T) = (0 -28270.086364563038) / Math.Sqrt((78606.66853418291 / (27)) + (173177.2802631066 / (27))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (27) - 2, .025) and -0.0661687113962734 = (26515.584318301775 - 28270.086364563038) / 26515.584318301775 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5da4a9e919dcee35f831ab69b6e475baaf798875
Compare dcda1e08cb8f421877388be2d685c009a31b1845
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.IterateFor(String).Span(Size%3a%20512).html>) 180.31 ns 235.63 ns 1.31 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 235.62683971517376 > 189.3199320644526. IsChangePoint: Marked as a change because one of 10/31/2022 4:38:04 PM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 1/5/2023 12:24:13 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -15.317179163320864 (T) = (0 -233.90327536024057) / Math.Sqrt((155.18706906029092 / (18)) + (84.63536473567798 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.27700499992621525 = (183.16551256553842 - 233.90327536024057) / 183.16551256553842 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5da4a9e919dcee35f831ab69b6e475baaf798875
Compare dcda1e08cb8f421877388be2d685c009a31b1845
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Backtracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options%3a%20None).html>) 1.26 μs 1.32 μs 1.05 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3239606796972447 > 1.3191189901070963. IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -20.814667956341378 (T) = (0 -1331.717167429668) / Math.Sqrt((10.880478490748978 / (18)) + (458.116848785917 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.06051874597370464 = (1255.7224212071474 - 1331.717167429668) / 1255.7224212071474 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5da4a9e919dcee35f831ab69b6e475baaf798875
Compare dcda1e08cb8f421877388be2d685c009a31b1845
Diff Diff

Regressions in Benchstone.BenchI.Array1

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/Benchstone.BenchI.Array1.Test.html>) 4.64 ms 4.99 ms 1.08 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Array1*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.Array1.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.988581795312499 > 4.908954842421875. IsChangePoint: Marked as a change because one of 10/31/2022 4:38:04 PM, 11/18/2022 8:48:12 PM, 12/1/2022 7:29:59 PM, 12/12/2022 5:25:39 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -10.730290212684308 (T) = (0 -4923707.661544424) / Math.Sqrt((6535449062.624477 / (18)) + (6782472906.592422 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.05393869536178307 = (4671721.119276557 - 4923707.661544424) / 4671721.119276557 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 5da4a9e919dcee35f831ab69b6e475baaf798875
Compare dcda1e08cb8f421877388be2d685c009a31b1845
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Widen_float - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Numerics.Tests.Perf_VectorConvert.Widen_float.html>) 876.35 ns 1.31 μs 1.50 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorConvert.Widen_float ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.314512369846482 > 920.5133007143095. IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 1/9/2023 8:29:54 AM, 1/17/2023 6:49:05 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 6:49:05 AM. IsRegressionStdDev: Marked as regression because -11.889841952582914 (T) = (0 -1277.7731000670874) / Math.Sqrt((10551.147027299177 / (18)) + (14999.990204966753 / (36))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (36) - 2, .025) and -0.4177794968895176 = (901.2495263688099 - 1277.7731000670874) / 901.2495263688099 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)