search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/arm64: 10 Improvements on 5/19/2023 4:22:12 AM #18006

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1ffe321285f2f16ea2eff5f623ef14a0b1b48249
Compare e5c3811c7628f4d5ae5ef0033163dd8cd1d52e17
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EnumerateActivityLinkTagsLarge - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsLarge.html>) 17.45 μs 6.87 μs 0.39 0.65 False
[ActivityAllocations - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat%3a%20W3C).html>) 347.60 ns 325.64 ns 0.94 0.09 False
[EnumerateActivityLinksLarge - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Diagnostics.Perf_Activity.EnumerateActivityLinksLarge.html>) 31.92 μs 8.25 μs 0.26 0.68 False
[EnumerateActivityEventsSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall.html>) 120.14 ns 38.91 ns 0.32 0.60 False
[EnumerateActivityTagObjectsLarge - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge.html>) 10.67 μs 6.87 μs 0.64 0.71 False
[EnumerateActivityLinksSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall.html>) 161.11 ns 61.20 ns 0.38 0.59 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsLarge ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.868809617077465 < 15.602092364759946. IsChangePoint: Marked as a change because one of 5/18/2023 7:10:13 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 13.367552806562676 (T) = (0 -7781.027933165009) / Math.Sqrt((12448274.690002041 / (299)) + (2004574.2870189887 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.512921614155502 = (15974.898823880758 - 7781.027933165009) / 15974.898823880758 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: W3C) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 325.63547024989106 < 331.281957143123. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 5.884659672169162 (T) = (0 -316.28560965640355) / Math.Sqrt((58.73137173790133 / (299)) + (201.96231145509816 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.105973196882582 = (353.7764287978107 - 316.28560965640355) / 353.7764287978107 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityLinksLarge ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 8.251692193283978 < 30.20493191009223. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 42.28860348619914 (T) = (0 -8277.041555142505) / Math.Sqrt((76280793.12814906 / (299)) + (2786.5601533915733 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.720936537111234 = (29660.06896589581 - 8277.041555142505) / 29660.06896589581 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 38.9091015930689 < 122.11251094232735. IsChangePoint: Marked as a change because one of 4/9/2023 11:24:15 AM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 48.66802514236267 (T) = (0 -43.67134253093236) / Math.Sqrt((609.1156307044935 / (299)) + (7.99094001104811 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.679981059722419 = (136.46486827639728 - 43.67134253093236) / 136.46486827639728 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 6.871584049192363 < 15.30986103892226. IsChangePoint: Marked as a change because one of 3/31/2023 7:02:21 PM, 5/14/2023 1:47:19 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 6.533971482271966 (T) = (0 -10820.260562707072) / Math.Sqrt((14077584.108935898 / (299)) + (12829734.726756942 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.3448565601438341 = (16515.86493040763 - 10820.260562707072) / 16515.86493040763 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 61.202540741081876 < 150.49157730286925. IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 18.93411955064582 (T) = (0 -54.89714481036869) / Math.Sqrt((950.0608588109176 / (299)) + (107.80697315616277 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.6543581469187877 = (158.82667078940239 - 54.89714481036869) / 158.82667078940239 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1ffe321285f2f16ea2eff5f623ef14a0b1b48249
Compare e5c3811c7628f4d5ae5ef0033163dd8cd1d52e17
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Formats.Cbor.Tests.Perf_CborWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteCoseKey - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey%3a%20ECDSA_P521).html>) 753.16 ns 664.64 ns 0.88 0.49 False
[WriteCoseKey - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey%3a%20ECDSA_P384).html>) 746.91 ns 581.34 ns 0.78 0.39 False
[WriteCoseKey - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey%3a%20ECDSA_P256).html>) 642.02 ns 514.21 ns 0.80 0.10 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Formats.Cbor.Tests.Perf_CborWriter*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P521) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 664.6371094348879 < 758.0842567404198. IsChangePoint: Marked as a change because one of 5/18/2023 7:10:13 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 8.899309712099905 (T) = (0 -660.4115155833043) / Math.Sqrt((1153.8160658746137 / (299)) + (1005.9800775233238 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1500086899563826 = (776.9626674764654 - 660.4115155833043) / 776.9626674764654 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P384) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 581.3376492352152 < 674.1308449189567. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 12.477593566088553 (T) = (0 -559.9476438425356) / Math.Sqrt((662.3932717623834 / (299)) + (727.3601774362339 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.21309661307144442 = (711.5837257075808 - 559.9476438425356) / 711.5837257075808 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Formats.Cbor.Tests.Perf_CborWriter.WriteCoseKey(publicKey: ECDSA_P256) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 514.207311598305 < 594.9626250622807. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 34.61480000962106 (T) = (0 -510.0442411541835) / Math.Sqrt((222.10618338221596 / (299)) + (50.958941361412705 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.19948480696133983 = (637.144985616221 - 510.0442411541835) / 637.144985616221 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1ffe321285f2f16ea2eff5f623ef14a0b1b48249
Compare e5c3811c7628f4d5ae5ef0033163dd8cd1d52e17
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.CreateAddAndClear(String).ImmutableSortedDictionary(Size%3a%20512).html>) 543.69 μs 497.61 μs 0.92 0.44 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<String>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 497.60657947716345 < 515.8050226697198. IsChangePoint: Marked as a change because one of 5/18/2023 7:10:13 PM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 6.871832081592065 (T) = (0 -485293.3112023989) / Math.Sqrt((465153538.18078387 / (299)) + (380160325.0319327 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10240553778626972 = (540659.8766279415 - 485293.3112023989) / 540659.8766279415 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
EgorBo commented 1 year ago

https://github.com/dotnet/runtime/pull/86225