search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/arm64: 16 Improvements on 5/16/2023 12:30:38 AM #18003

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 4c0f2e7bd4cb86917e362052e0642df600984b6c
Compare 522c281d5e2ccbfa9a114018a7309965571ab4b6
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Boolean.TryParse(value%3a%20%22TRUE%22).html>) 2.27 ns 0.03 ns 0.01 0.67 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Boolean.Parse(value%3a%20%22False%22).html>) 1.65 ns 0.35 ns 0.21 0.49 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Boolean.Parse(value%3a%20%22false%22).html>) 1.77 ns 0.00 ns 0.00 0.49 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Boolean.TryParse(value%3a%20%22true%22).html>) 2.03 ns 0.00 ns 0.00 0.63 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Boolean.TryParse(value%3a%20%22%20True%20%22).html>) 8.54 ns 6.47 ns 0.76 0.28 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Boolean.TryParse(value%3a%20%22False%22).html>) 2.25 ns 0.05 ns 0.02 0.49 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Boolean.Parse(value%3a%20%22TRUE%22).html>) 1.41 ns 0.00 ns 0.00 0.65 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Boolean.TryParse(value%3a%20%22false%22).html>) 2.51 ns 0.51 ns 0.20 0.51 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Boolean.Parse(value%3a%20%22%20True%20%22).html>) 9.21 ns 6.74 ns 0.73 0.20 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Boolean.TryParse(value%3a%20%22Bogus%22).html>) 4.98 ns 3.94 ns 0.79 0.41 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Boolean*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Boolean.TryParse(value: "TRUE") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0.03301557590400751 < 2.0118784060437394. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/15/2023 5:11:34 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 18.00200062176273 (T) = (0 -0.10430799527382131) / Math.Sqrt((0.2646886362651754 / (299)) + (0.11942531988168278 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.9419718971329486 = (1.7975427442941252 - 0.10430799527382131) / 1.7975427442941252 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Boolean.Parse(value: "False") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0.35119569986919036 < 1.65063824477238. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/15/2023 5:11:34 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 31.602189060890517 (T) = (0 -0.27142150708120616) / Math.Sqrt((0.06313032025334635 / (299)) + (0.043086130492378936 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.8660494497365372 = (2.0262813892690725 - 0.27142150708120616) / 2.0262813892690725 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Boolean.Parse(value: "false") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0 < 1.5131424848602997. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/15/2023 5:11:34 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 51.31092677135859 (T) = (0 -0.24707486385131047) / Math.Sqrt((0.05741549548235826 / (299)) + (0.014945499747594878 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.8774381245320422 = (2.0159194113825794 - 0.24707486385131047) / 2.0159194113825794 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Boolean.TryParse(value: "true") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0 < 1.8768574596947107. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/15/2023 5:11:34 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 96.23518822454328 (T) = (0 -0.020202820570751137) / Math.Sqrt((0.06703698304060143 / (299)) + (0.00168538006403601 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.9886863252530801 = (1.7856992553415356 - 0.020202820570751137) / 1.7856992553415356 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Boolean.TryParse(value: " True ") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.470302394900701 < 8.259747317937011. IsChangePoint: Marked as a change because one of 2/6/2023 4:57:06 PM, 4/25/2023 6:46:56 PM, 5/15/2023 5:11:34 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 8.955772857720653 (T) = (0 -6.894381348211003) / Math.Sqrt((0.37844767228562526 / (299)) + (1.056294774807232 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.26459222886747896 = (9.374909565605652 - 6.894381348211003) / 9.374909565605652 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Boolean.TryParse(value: "False") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0.053609110961034885 < 2.1330050716126667. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/15/2023 5:11:34 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 24.388164288171414 (T) = (0 -0.42372061922832005) / Math.Sqrt((0.10052488086077493 / (299)) + (0.08345262777916543 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.8203832673256917 = (2.359026427658247 - 0.42372061922832005) / 2.359026427658247 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Boolean.Parse(value: "TRUE") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0 < 1.148770749253362. IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 25.082337319872988 (T) = (0 -0.15991786632278118) / Math.Sqrt((0.04715082799163767 / (299)) + (0.03544965211180372 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.8873256304932707 = (1.4192923113115836 - 0.15991786632278118) / 1.4192923113115836 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Boolean.TryParse(value: "false") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0.5096569136660126 < 2.2174917318659007. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/15/2023 9:18:21 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 38.57246418486124 (T) = (0 -0.30025986142647204) / Math.Sqrt((0.08042398724106209 / (299)) + (0.04075280166784432 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.8720750189869527 = (2.3471558021638317 - 0.30025986142647204) / 2.3471558021638317 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Boolean.Parse(value: " True ") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.7394198411169715 < 8.770909186852066. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 5:11:34 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 21.61401671796599 (T) = (0 -7.23385804221678) / Math.Sqrt((0.25818246118533783 / (299)) + (0.21934402150972326 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.27104516127955225 = (9.92360247572339 - 7.23385804221678) / 9.92360247572339 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_Boolean.TryParse(value: "Bogus") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.9438450149233217 < 4.777288552766861. IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 4.7722832860628035 (T) = (0 -4.249530529854663) / Math.Sqrt((0.20226880522675753 / (299)) + (0.5450963712566265 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.17767452794098346 = (5.167699012429088 - 4.249530529854663) / 5.167699012429088 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 4c0f2e7bd4cb86917e362052e0642df600984b6c
Compare 522c281d5e2ccbfa9a114018a7309965571ab4b6
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22.%7b2%2c4%7d(Tom%7cSawyer%7cHuckleberry%7cFinn)%22%2c%20Options%3a%20Compiled).html>) 606.94 ms 492.26 ms 0.81 0.01 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22Tom%7cSawyer%7cHuckleberry%7cFinn%22%2c%20Options%3a%20None).html>) 21.62 ms 18.09 ms 0.84 0.01 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22.%7b2%2c4%7d(Tom%7cSawyer%7cHuckleberry%7cFinn)%22%2c%20Options%3a%20None).html>) 7.08 secs 5.45 secs 0.77 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22.%7b0%2c2%7d(Tom%7cSawyer%7cHuckleberry%7cFinn)%22%2c%20Options%3a%20None).html>) 7.07 secs 5.43 secs 0.77 0.00 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22(%5bA-Za-z%5dawyer%7c%5bA-Za-z%5dinn)%5c%5cs%22%2c%20Options%3a%20None).html>) 1.31 secs 1.14 secs 0.87 0.03 True
[Count - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern%3a%20%22.%7b0%2c2%7d(Tom%7cSawyer%7cHuckleberry%7cFinn)%22%2c%20Options%3a%20Compiled).html>) 598.40 ms 492.00 ms 0.82 0.00 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 492.2570242 < 576.4638681181318. IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 56.02217676761231 (T) = (0 -493961664.49714285) / Math.Sqrt((39744865315691.01 / (299)) + (47001515129964.64 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.17010657585834724 = (595210963.3933303 - 493961664.49714285) / 595210963.3933303 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom|Sawyer|Huckleberry|Finn", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.08890955357143 < 20.631687048854165. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 58.358347356752674 (T) = (0 -18166230.645073257) / Math.Sqrt((21557200178.610046 / (299)) + (19168283868.931644 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.16719841979214423 = (21813395.983877957 - 18166230.645073257) / 21813395.983877957 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.44524518025 < 6.699383383911905. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 435.092467805601 (T) = (0 -5446044582.163333) / Math.Sqrt((3263015274597308 / (299)) + (15247628972434.068 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.22989673453715231 = (7071836760.606579 - 5446044582.163333) / 7071836760.606579 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.432366628333333 < 6.755135484565358. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 296.2015370415698 (T) = (0 -5443406306.801905) / Math.Sqrt((1553028648568799.2 / (299)) + (129442969023856.69 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.23275942411183623 = (7094784188.7801285 - 5443406306.801905) / 7094784188.7801285 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "([A-Za-z]awyer|[A-Za-z]inn)\\s", Options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1375537912142857 < 1.245059414100238. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 74.32483500590185 (T) = (0 -1140277832.55685) / Math.Sqrt((143940610240982.53 / (299)) + (24632207234557.23 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13162616983186798 = (1313118605.0782676 - 1140277832.55685) / 1313118605.0782676 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 492.0034635714286 < 570.7479855996794. IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 60.07576394098376 (T) = (0 -495354455.4199756) / Math.Sqrt((30383462800481.82 / (299)) + (37842408701524.36 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.16420983189783095 = (592678012.1675495 - 495354455.4199756) / 592678012.1675495 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
EgorBo commented 1 year ago

Perf_Boolean is https://github.com/dotnet/runtime/pull/86129