search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 23 Improvements on 6/9/2023 5:38:58 AM #18807

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 3fded0bb77c852b10d712bd629ced740499a3aa6
Compare fa0d21579125aaf954e1e0d5084842d69feba299
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Int32.ToStringHex(value%3a%20-2147483648).html>) 19.85 ns 17.49 ns 0.88 0.13 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int32.ToStringHex(value: -2147483648) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.492068430494417 < 18.714462525328067. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 11.733659640863195 (T) = (0 -16.682338653674595) / Math.Sqrt((3.185644768767032 / (299)) + (0.27321217442997764 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.1139352954544804 = (18.827449697628236 - 16.682338653674595) / 18.827449697628236 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 3fded0bb77c852b10d712bd629ced740499a3aa6
Compare fa0d21579125aaf954e1e0d5084842d69feba299
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 687.11 ns 416.13 ns 0.61 0.19 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20None%2c%20False)).html>) 550.69 ns 377.45 ns 0.69 0.13 False
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 497.56 ns 449.68 ns 0.90 0.19 False
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreNonSpace%2c%20False)).html>) 558.43 ns 404.64 ns 0.72 0.07 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreNonSpace%2c%20False)).html>) 550.84 ns 372.20 ns 0.68 0.13 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(%2c%20None%2c%20False)).html>) 407.97 ns 374.54 ns 0.92 0.10 False
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(%2c%20None%2c%20False)).html>) 557.09 ns 405.66 ns 0.73 0.09 False
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20None%2c%20False)).html>) 558.01 ns 404.71 ns 0.73 0.09 False
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 498.44 ns 449.91 ns 0.90 0.17 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 688.21 ns 432.61 ns 0.63 0.19 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 416.1314182566085 < 653.2247253660831. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 20.290578850003634 (T) = (0 -420.0419829551738) / Math.Sqrt((14088.325859853507 / (299)) + (52.34579134352311 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.2573862051144768 = (565.6264209580498 - 420.0419829551738) / 565.6264209580498 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, None, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 377.45356879817183 < 455.290764753275. IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/26/2023 5:44:43 PM, 5/29/2023 5:51:10 AM, 5/30/2023 10:12:23 PM, 6/4/2023 12:53:30 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 10.333967036978368 (T) = (0 -369.1427356444749) / Math.Sqrt((1872.4863351003667 / (299)) + (149.20278443459506 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.08578754540519795 = (403.78222128693994 - 369.1427356444749) / 403.78222128693994 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 449.6825741719114 < 472.57038950135694. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 6/5/2023 4:27:01 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 17.618152568790375 (T) = (0 -452.53414063436435) / Math.Sqrt((9207.75703517536 / (299)) + (12.634036343244775 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.18013310017779532 = (551.9604959445251 - 452.53414063436435) / 551.9604959445251 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 404.63780681897043 < 529.9084930643335. IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/5/2023 4:27:01 PM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 5.685869168662594 (T) = (0 -387.92043990791365) / Math.Sqrt((2941.309690685143 / (299)) + (391.13286919948877 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.06813039216082052 = (416.2818882004577 - 387.92043990791365) / 416.2818882004577 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 372.2019209334313 < 455.3871124458842. IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/29/2023 7:53:51 AM, 6/5/2023 4:27:01 PM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 13.638743183321475 (T) = (0 -366.95457477501157) / Math.Sqrt((1520.3266327419797 / (299)) + (34.025473540694215 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.08590501398596107 = (401.4403102407738 - 366.95457477501157) / 401.4403102407738 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, None, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 374.5355915361002 < 455.3190345409975. IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/26/2023 5:44:43 PM, 5/29/2023 1:48:21 PM, 6/1/2023 1:04:59 AM, 6/4/2023 9:29:52 PM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 13.397380153718817 (T) = (0 -365.4237516476278) / Math.Sqrt((2005.2032721334274 / (299)) + (36.57783514934321 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09402222943787349 = (403.3473706765405 - 365.4237516476278) / 403.3473706765405 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 405.6560442908945 < 529.5654210411495. IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/5/2023 4:27:01 PM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 5.670113522185328 (T) = (0 -387.9203413675273) / Math.Sqrt((2939.467128497667 / (299)) + (395.6815267079086 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.0681688660360029 = (416.29897009055645 - 387.9203413675273) / 416.29897009055645 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 404.710023632472 < 530.1638459336816. IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/5/2023 4:27:01 PM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 6.043302648337522 (T) = (0 -387.1734829885863) / Math.Sqrt((2851.1958675697424 / (299)) + (402.7857412824406 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.07243092125708765 = (417.4066297178675 - 387.1734829885863) / 417.4066297178675 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 449.9098567606463 < 472.7577332799532. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 17.970517068951956 (T) = (0 -450.5732053267015) / Math.Sqrt((9331.840110617739 / (299)) + (3.8793645627983167 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.1829834446416752 = (551.4860162522538 - 450.5732053267015) / 551.4860162522538 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 432.60827401848314 < 653.7357909386145. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 20.023729988039197 (T) = (0 -421.01211164375604) / Math.Sqrt((14760.71291582827 / (299)) + (56.85593332064187 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.25917116338315144 = (568.298763269525 - 421.01211164375604) / 568.298763269525 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 3fded0bb77c852b10d712bd629ced740499a3aa6
Compare fa0d21579125aaf954e1e0d5084842d69feba299
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LinqOrderByExtension - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Collections.Sort(BigStruct).LinqOrderByExtension(Size%3a%20512).html>) 29.54 μs 18.49 μs 0.63 0.23 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.487368826666668 < 27.009776537967035. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 40.522952514313786 (T) = (0 -18166.132856343105) / Math.Sqrt((36642188.52844713 / (299)) + (103996.60083575067 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.4469139346506773 = (32845.03804099564 - 18166.132856343105) / 32845.03804099564 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 3fded0bb77c852b10d712bd629ced740499a3aa6
Compare fa0d21579125aaf954e1e0d5084842d69feba299
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Perf_Utf8Encoding.GetString(Input%3a%20Chinese).html>) 186.67 μs 169.01 μs 0.91 0.29 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Perf_Utf8Encoding.GetString(Input: Chinese) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 169.01098781370658 < 182.08180411331298. IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/11/2023 5:02:01 PM, 6/5/2023 4:27:01 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 30.295917310816954 (T) = (0 -170519.28922648547) / Math.Sqrt((243838168.0493432 / (299)) + (2988335.285137189 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.1549145628007817 = (201777.57386474486 - 170519.28922648547) / 201777.57386474486 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 3fded0bb77c852b10d712bd629ced740499a3aa6
Compare fa0d21579125aaf954e1e0d5084842d69feba299
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteAsciiCharArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars%3a%208000).html>) 469.61 ns 365.10 ns 0.78 0.03 False
[WriteAsciiCharArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars%3a%202000000).html>) 115.72 μs 107.05 μs 0.93 0.03 False
[WriteAsciiString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars%3a%208000).html>) 466.94 ns 361.74 ns 0.77 0.01 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 365.0974673262786 < 446.345743905603. IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 8.544859670612684 (T) = (0 -364.6120328026211) / Math.Sqrt((1561.1252677492905 / (299)) + (2.8284071196543215 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.05190361975438459 = (384.57275061862816 - 364.6120328026211) / 384.57275061862816 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 2000000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 107.05462514717422 < 109.36537981387869. IsChangePoint: Marked as a change because one of 4/29/2023 8:32:21 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 6/5/2023 4:27:01 PM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 17.22274621594936 (T) = (0 -106889.88059617701) / Math.Sqrt((8818551.869273558 / (299)) + (2108638.5019986285 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.0508586005157854 = (112617.44630912048 - 106889.88059617701) / 112617.44630912048 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 361.7448004177264 < 445.87291693352. IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 8.951540132900954 (T) = (0 -365.0762737936568) / Math.Sqrt((1455.037257781785 / (299)) + (4.660693258497146 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.053215803455499526 = (385.5960789439493 - 365.0762737936568) / 385.5960789439493 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 3fded0bb77c852b10d712bd629ced740499a3aa6
Compare fa0d21579125aaf954e1e0d5084842d69feba299
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParseUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value%3a%2018446744073709551615).html>) 21.88 ns 15.68 ns 0.72 0.05 False
[TryParseUInt64Hex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value%3a%203039).html>) 5.82 ns 4.48 ns 0.77 0.25 False
[TryParseUInt64Hex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value%3a%20FFFFFFFFFFFFFFFF).html>) 19.02 ns 11.32 ns 0.60 0.24 False
[TryParseInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value%3a%20-9223372036854775808).html>) 22.01 ns 15.38 ns 0.70 0.17 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.67782476033963 < 20.88116681232263. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 20.683295012800073 (T) = (0 -15.702166953660507) / Math.Sqrt((2.4746439482227527 / (299)) + (0.1626702991747175 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.16292064113111843 = (18.758277560300066 - 15.702166953660507) / 18.758277560300066 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 3039) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.481031396760735 < 5.532140226363697. IsChangePoint: Marked as a change because one of 5/11/2023 5:02:01 PM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 12.257592093559628 (T) = (0 -4.560314121780747) / Math.Sqrt((0.2089284589960979 / (299)) + (0.021597592726959058 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.11844204250160664 = (5.1730167971276675 - 4.560314121780747) / 5.1730167971276675 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: FFFFFFFFFFFFFFFF) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.321280239128836 < 17.695259099355823. IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 10.70789845774254 (T) = (0 -10.673809423448404) / Math.Sqrt((7.426719908994785 / (299)) + (0.15058932424171176 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.16246248321018433 = (12.74427617805096 - 10.673809423448404) / 12.74427617805096 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.381879220735314 < 20.553717049830784. IsChangePoint: Marked as a change because one of 3/19/2023 10:05:03 PM, 3/25/2023 12:18:49 AM, 4/9/2023 11:04:55 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 21.392036004227407 (T) = (0 -15.285729899068775) / Math.Sqrt((9.163106880424332 / (299)) + (0.012955772494315638 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.19953256228399943 = (19.096004632848075 - 15.285729899068775) / 19.096004632848075 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 3fded0bb77c852b10d712bd629ced740499a3aa6
Compare fa0d21579125aaf954e1e0d5084842d69feba299
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Compare_Same - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Globalization.Tests.StringEquality.Compare_Same(Count%3a%201024%2c%20Options%3a%20(en-US%2c%20OrdinalIgnoreCase)).html>) 679.94 ns 531.02 ns 0.78 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringEquality*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, OrdinalIgnoreCase)) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 531.0156532241573 < 644.3958357914639. IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/10/2023 6:18:38 PM, 5/11/2023 1:19:27 PM, 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 10.617726283020128 (T) = (0 -532.5572734497617) / Math.Sqrt((11783.834040015256 / (299)) + (2.273692803060439 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.11147649661123314 = (599.3733102372929 - 532.5572734497617) / 599.3733102372929 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 3fded0bb77c852b10d712bd629ced740499a3aa6
Compare fa0d21579125aaf954e1e0d5084842d69feba299
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Guid.Parse.html>) 25.51 ns 21.16 ns 0.83 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Guid*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Guid.Parse ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 21.164606165762386 < 24.16976450222895. IsChangePoint: Marked as a change because one of 4/1/2023 1:31:30 PM, 4/25/2023 6:46:56 PM, 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 4.915436796153942 (T) = (0 -21.464132105174855) / Math.Sqrt((3.1817548856090303 / (299)) + (0.5089769431186346 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.0501061880788648 = (22.596349019017413 - 21.464132105174855) / 22.596349019017413 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 3fded0bb77c852b10d712bd629ced740499a3aa6
Compare fa0d21579125aaf954e1e0d5084842d69feba299
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IndexOfAnyFourValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.Span(Byte).IndexOfAnyFourValues(Size%3a%204).html>) 5.26 ns 4.04 ns 0.77 0.24 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 4) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 4.042795302133654 < 4.998227232901489. IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 4/11/2023 10:53:42 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM. IsImprovementStdDev: Marked as improvement because 52.66190156942751 (T) = (0 -4.042190583466297) / Math.Sqrt((0.10546545023740463 / (299)) + (1.3572545550545672E-05 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.19683387038821212 = (5.032820028678374 - 4.042190583466297) / 5.032820028678374 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)