dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x86: 1022 Improvements on 5/19/2023 3:32:16 PM #18161

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToStream(Mode%3a%20Reflection).html>) 6.97 μs 6.12 μs 0.88 0.03 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToWriter(Mode%3a%20Reflection).html>) 6.59 μs 5.68 μs 0.86 0.01 True
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 6.69 μs 6.06 μs 0.91 0.02 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.120463126554116 < 6.671271261603767. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 21.818393589760788 (T) = (0 -6189.443546481246) / Math.Sqrt((16223.147413451072 / (299)) + (18741.800761566967 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.1129093482052775 = (6977.239061147852 - 6189.443546481246) / 6977.239061147852 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.676938552265019 < 6.2701201747665225. IsChangePoint: Marked as a change because one of 5/12/2023 4:02:48 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 59.15091790804757 (T) = (0 -5722.22756750624) / Math.Sqrt((23241.149607072446 / (299)) + (5309.227376527465 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.17679982137414552 = (6951.1981606444715 - 5722.22756750624) / 6951.1981606444715 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.060096041909198 < 6.376577690050807. IsChangePoint: Marked as a change because one of 5/12/2023 4:02:48 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 73.24275117591152 (T) = (0 -6087.547382167457) / Math.Sqrt((27815.06659622628 / (299)) + (1429.2634773367677 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.14170830187086242 = (7092.632254787965 - 6087.547382167457) / 7092.632254787965 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x86
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare 558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TrimEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.TrimEnd(s%3a%20%22Test%22).html>) 4.07 ns 2.34 ns 0.58 0.10 False
[Format_MultipleArgs - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.Format_MultipleArgs.html>) 298.02 ns 274.21 ns 0.92 0.01 False
[TrimStart_CharArr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.TrimStart_CharArr(s%3a%20%22Test%22%2c%20c%3a%20%5b%27%20%27%2c%20%27%e2%80%85%27%5d).html>) 8.34 ns 6.11 ns 0.73 0.04 False
[TrimStart - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.TrimStart(s%3a%20%22Test%22).html>) 4.99 ns 2.28 ns 0.46 0.10 False
[TrimEnd_CharArr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.TrimEnd_CharArr(s%3a%20%22Test%22%2c%20c%3a%20%5b%27%20%27%2c%20%27%e2%80%85%27%5d).html>) 8.17 ns 6.42 ns 0.79 0.02 False
[Trim_CharArr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.Trim_CharArr(s%3a%20%22%20Test%22%2c%20c%3a%20%5b%27%20%27%2c%20%27%e2%80%85%27%5d).html>) 18.82 ns 17.30 ns 0.92 0.04 False
[Trim_CharArr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.Trim_CharArr(s%3a%20%22Test%20%22%2c%20c%3a%20%5b%27%20%27%2c%20%27%e2%80%85%27%5d).html>) 18.59 ns 17.25 ns 0.93 0.05 False
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.Trim(s%3a%20%22%20Te%20st%20%20%22).html>) 18.20 ns 13.10 ns 0.72 0.04 False
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.Trim(s%3a%20%22%20Test%22).html>) 16.07 ns 10.55 ns 0.66 0.05 False
[Concat_CharEnumerable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.Concat_CharEnumerable.html>) 7.93 μs 3.78 μs 0.48 0.04 False
[Split - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.Split(s%3a%20%22A%20B%20C%20D%20E%20F%20G%20H%20I%20J%20K%20L%20M%20N%20O%20P%20Q%20R%20S%20T%20U%20V%20W%20X%20Y%20Z%22%2c%20arr%3a%20%5b%27%20%27%5d%2c%20options%3a%20None).html>) 409.71 ns 363.83 ns 0.89 0.03 False
[TrimStart_CharArr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.TrimStart_CharArr(s%3a%20%22%20Test%22%2c%20c%3a%20%5b%27%20%27%2c%20%27%e2%80%85%27%5d).html>) 17.41 ns 15.21 ns 0.87 0.07 True
[TrimStart - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.TrimStart(s%3a%20%22%20Test%22).html>) 13.81 ns 10.23 ns 0.74 0.07 True
[TrimEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.TrimEnd(s%3a%20%22Test%20%22).html>) 13.04 ns 10.24 ns 0.79 0.13 False
[IndexOfAny - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.IndexOfAny.html>) 11.37 ns 9.05 ns 0.80 0.04 False
[TrimEnd_CharArr - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.TrimEnd_CharArr(s%3a%20%22Test%20%22%2c%20c%3a%20%5b%27%20%27%2c%20%27%e2%80%85%27%5d).html>) 17.76 ns 16.55 ns 0.93 0.07 False
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.Trim(s%3a%20%22Test%20%22).html>) 16.65 ns 12.76 ns 0.77 0.10 False
[Split - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.Split(s%3a%20%22ABCDEFGHIJKLMNOPQRSTUVWXYZ%22%2c%20arr%3a%20%5b%27%20%27%5d%2c%20options%3a%20RemoveEmptyEntries).html>) 26.60 ns 22.49 ns 0.85 0.07 False
[Join_Enumerable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x86_Windows 10.0.18362/System.Tests.Perf_String.Join_Enumerable.html>) 328.56 ns 303.67 ns 0.92 0.02 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_String.TrimEnd(s: "Test") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.3438985623792212 < 3.869659557295686. IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 23.96425238765755 (T) = (0 -2.0867844082137355) / Math.Sqrt((0.47098055634386393 / (299)) + (0.005450756560408324 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.335823202066223 = (3.1419110313784286 - 2.0867844082137355) / 3.1419110313784286 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Format_MultipleArgs ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 274.210593948847 < 283.19231353052794. IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 54.37707984771145 (T) = (0 -274.1208724042948) / Math.Sqrt((11.750358533554136 / (299)) + (1.9694356523286645 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.07572794924106648 = (296.5802895145537 - 274.1208724042948) / 296.5802895145537 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.TrimStart_CharArr(s: "Test", c: [' ', ' ']) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.106920741429989 < 7.924927293541374. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 5.714990932342782 (T) = (0 -6.0963672354761105) / Math.Sqrt((1.6455362939875104 / (299)) + (0.02019686959076991 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.07200042716752017 = (6.569364269068054 - 6.0963672354761105) / 6.569364269068054 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.TrimStart(s: "Test") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.276191808955187 < 4.753950440525916. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 16.24501879588042 (T) = (0 -2.0187038528360417) / Math.Sqrt((1.6231598276629506 / (299)) + (0.005534126679673089 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.37993419738474377 = (3.2556284257602 - 2.0187038528360417) / 3.2556284257602 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.TrimEnd_CharArr(s: "Test", c: [' ', ' ']) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.4188204178091 < 7.766670531673097. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 8.096183066287008 (T) = (0 -6.329014138164044) / Math.Sqrt((0.9937146646587903 / (299)) + (0.000813879670464093 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.0692009338294976 = (6.799549299294961 - 6.329014138164044) / 6.799549299294961 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Trim_CharArr(s: " Test", c: [' ', ' ']) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.297533466077077 < 17.81951184567007. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 8:24:48 PM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 48.81215594905538 (T) = (0 -17.241606264948686) / Math.Sqrt((0.6408180949280236 / (299)) + (0.009587728397857105 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.13190682733667478 = (19.861469722254736 - 17.241606264948686) / 19.861469722254736 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' ']) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.248446434866107 < 17.550492977451988. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 21.305546879373985 (T) = (0 -17.053286996471847) / Math.Sqrt((0.9519853551990178 / (299)) + (0.2079092244537368 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.14023506100978128 = (19.83482487259911 - 17.053286996471847) / 19.83482487259911 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Trim(s: " Te st ") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.10226830913381 < 17.266463988254124. IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 29.10847596399421 (T) = (0 -13.024978153315784) / Math.Sqrt((0.7833877223673329 / (299)) + (0.4494425852025699 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2874453446364976 = (18.27926890277803 - 13.024978153315784) / 18.27926890277803 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Trim(s: " Test") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.550499745657556 < 15.170409700804306. IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 92.91837836152722 (T) = (0 -10.378306196198498) / Math.Sqrt((0.6980282283542999 / (299)) + (0.017073589080528962 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.3453831119476915 = (15.854015357100286 - 10.378306196198498) / 15.854015357100286 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Concat_CharEnumerable ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.7773952525252534 < 7.536990402502959. IsChangePoint: Marked as a change because one of 4/4/2023 11:36:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 13.736461556592007 (T) = (0 -5823.030530863309) / Math.Sqrt((360610.6032210548 / (299)) + (320437.80958765326 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.26165864250403065 = (7886.6373551275665 - 5823.030530863309) / 7886.6373551275665 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Split(s: "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z", arr: [' '], options: None) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 363.83080828929985 < 389.2120477215968. IsChangePoint: Marked as a change because one of 3/22/2023 7:51:01 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 20.953552044087157 (T) = (0 -369.02473024299263) / Math.Sqrt((226.49389245489283 / (299)) + (149.57197418071797 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.15682001163227424 = (437.6583117886503 - 369.02473024299263) / 437.6583117886503 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.TrimStart_CharArr(s: " Test", c: [' ', ' ']) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 15.208424813664408 < 16.537159645762717. IsChangePoint: Marked as a change because one of 5/2/2023 12:34:35 AM, 5/19/2023 3:32:16 PM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 27.95486079365452 (T) = (0 -14.873694359919233) / Math.Sqrt((0.12391386290937861 / (299)) + (0.15624398244447724 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.16819565237748993 = (17.881241427063593 - 14.873694359919233) / 17.881241427063593 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.TrimStart(s: " Test") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.227841475306672 < 12.808152093142713. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 46.86954780246596 (T) = (0 -10.143381237822416) / Math.Sqrt((0.1788747974560245 / (299)) + (0.07763098929874135 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2598632819145005 = (13.704739935157043 - 10.143381237822416) / 13.704739935157043 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.TrimEnd(s: "Test ") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 10.243852229695653 < 12.37542323143053. IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 21.292078684476458 (T) = (0 -10.253866576875696) / Math.Sqrt((0.22110741521055383 / (299)) + (0.35302550498203283 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.24444103707254794 = (13.57123279584503 - 10.253866576875696) / 13.57123279584503 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.IndexOfAny ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.045402249281116 < 10.795244648462333. IsChangePoint: Marked as a change because one of 3/28/2023 3:45:48 AM, 4/25/2023 6:46:56 PM, 5/11/2023 9:33:40 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 24.28909663333257 (T) = (0 -9.115790035457342) / Math.Sqrt((0.4731802111449823 / (299)) + (0.009430288181171692 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.11134416141229793 = (10.257953236367218 - 9.115790035457342) / 10.257953236367218 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' ']) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 16.548936039408833 < 16.87342027764877. IsChangePoint: Marked as a change because one of 4/27/2023 1:05:59 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 12.688624532492412 (T) = (0 -15.895233129221374) / Math.Sqrt((0.2442442684871065 / (299)) + (0.473414039591829 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.12559794288946372 = (18.178403172731787 - 15.895233129221374) / 18.178403172731787 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Trim(s: "Test ") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.760561734577333 < 15.569397173448044. IsChangePoint: Marked as a change because one of 4/4/2023 8:16:00 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 46.39607872563722 (T) = (0 -12.597955356308818) / Math.Sqrt((1.2162525893677014 / (299)) + (0.014663485967737035 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2073508097401669 = (15.893481644987444 - 12.597955356308818) / 15.893481644987444 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: RemoveEmptyEntries) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.48777248294544 < 25.29174924171048. IsChangePoint: Marked as a change because one of 5/10/2023 6:05:58 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 78.57518870780584 (T) = (0 -22.486492487407627) / Math.Sqrt((1.166006462301506 / (299)) + (0.04780520895010916 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2272988873655496 = (29.101151945727224 - 22.486492487407627) / 29.101151945727224 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_String.Join_Enumerable ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 303.66825444137027 < 312.0273665672371. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 11:54:24 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM. IsImprovementStdDev: Marked as improvement because 22.669612109413357 (T) = (0 -301.0854389536597) / Math.Sqrt((201.57018175063368 / (299)) + (79.49099416152298 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.15542132413189955 = (356.4918787988448 - 301.0854389536597) / 356.4918787988448 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
kunalspathak commented 1 year ago

https://github.com/dotnet/runtime/pull/86225