search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Changes at 4/18/2022 10:15:32 PM #4814

Closed performanceautofiler[bot] closed 1 year ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.Collections.Tests.DictionarySequentialKeys

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryGetValue_3k_Int_Int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_Int.html>) 67.31 ns 21.62 ns 0.32 0.74 False
[ContainsKey_3k_Int_32ByteRefsValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.ContainsKey_3k_Int_32ByteRefsValue.html>) 88.66 ns 22.83 ns 0.26 0.60 False
[ContainsValue_3k_Int_Int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.ContainsValue_3k_Int_Int.html>) 88.53 ns 21.42 ns 0.24 0.59 False
[TryGetValue_17_Int_Int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_Int.html>) 67.75 ns 22.22 ns 0.33 0.72 False
[ContainsValue_17_Int_Int - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.ContainsValue_17_Int_Int.html>) 54.29 ns 21.43 ns 0.39 0.59 False
[TryGetValue_17_Int_32ByteValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteValue.html>) 65.90 ns 24.84 ns 0.38 0.62 False
[ContainsKey_17_Int_32ByteRefsValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.ContainsKey_17_Int_32ByteRefsValue.html>) 52.75 ns 21.88 ns 0.41 0.62 False
[TryGetValue_17_Int_32ByteRefsValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteRefsValue.html>) 71.41 ns 34.01 ns 0.48 0.43 True
[ContainsKey_17_Int_32ByteValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.ContainsKey_17_Int_32ByteValue.html>) 88.80 ns 21.85 ns 0.25 0.60 False
[TryGetValue_3k_Int_32ByteRefsValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteRefsValue.html>) 71.11 ns 33.46 ns 0.47 0.48 False
[TryGetValue_3k_Int_32ByteValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteValue.html>) 89.37 ns 22.84 ns 0.26 0.66 False
[ContainsKey_3k_Int_32ByteValue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.DictionarySequentialKeys.ContainsKey_3k_Int_32ByteValue.html>) 53.40 ns 22.62 ns 0.42 0.61 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_Int ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.623358951580283 < 67.64964040918802. IsChangePoint: Marked as a change because one of 3/18/2022 2:15:20 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 62.691258072673044 (T) = (0 -22.471569485510727) / Math.Sqrt((79.23131941903327 / (299)) + (1.949942686424275 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.6203272300305228 = (59.186676693504445 - 22.471569485510727) / 59.186676693504445 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.DictionarySequentialKeys.ContainsKey_3k_Int_32ByteRefsValue ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 22.83441631032927 < 50.762994680639906. IsChangePoint: Marked as a change because one of 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 64.0805731672495 (T) = (0 -21.91153016397153) / Math.Sqrt((106.27387439467869 / (299)) + (0.14531054711439825 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.6373837183423758 = (60.42621711250133 - 21.91153016397153) / 60.42621711250133 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.DictionarySequentialKeys.ContainsValue_3k_Int_Int ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.421903107393856 < 51.54743147952713. IsChangePoint: Marked as a change because one of 4/7/2022 2:02:15 PM, 4/13/2022 10:02:29 PM, 4/18/2022 8:59:00 AM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 56.202155814604 (T) = (0 -21.17594849392161) / Math.Sqrt((143.40220249392004 / (299)) + (0.12424187964656465 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.6488181459240863 = (60.299096459989876 - 21.17594849392161) / 60.299096459989876 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_Int ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 22.217347568075137 < 63.33233680697996. IsChangePoint: Marked as a change because one of 4/7/2022 6:53:49 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 60.41812465118465 (T) = (0 -23.00711208984896) / Math.Sqrt((107.32440871837082 / (299)) + (0.6457036193350679 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.6196197175342743 = (60.48450235304197 - 23.00711208984896) / 60.48450235304197 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.DictionarySequentialKeys.ContainsValue_17_Int_Int ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.426949397104384 < 51.43441939568699. IsChangePoint: Marked as a change because one of 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 90.29953655318427 (T) = (0 -21.70174322490423) / Math.Sqrt((43.51583752374182 / (299)) + (0.1697897905549879 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.6189007543895343 = (56.945122497267555 - 21.70174322490423) / 56.945122497267555 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteValue ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 24.8437689521719 < 73.698513101047. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 58.862103926541465 (T) = (0 -23.702945117291964) / Math.Sqrt((101.35142757174496 / (299)) + (0.4449788593014223 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.5973083122588945 = (58.86127238000211 - 23.702945117291964) / 58.86127238000211 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.DictionarySequentialKeys.ContainsKey_17_Int_32ByteRefsValue ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.884147696429004 < 50.322049648958156. IsChangePoint: Marked as a change because one of 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 56.66426921281104 (T) = (0 -22.389975612612307) / Math.Sqrt((122.35626079419525 / (299)) + (0.44470150580393186 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.6231763574197909 = (59.41765081220049 - 22.389975612612307) / 59.41765081220049 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_32ByteRefsValue ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 34.00792239704591 < 67.91664897751221. IsChangePoint: Marked as a change because one of 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 125.28330928429715 (T) = (0 -33.682080569830944) / Math.Sqrt((26.953945538080877 / (299)) + (0.25749855766956636 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.541044617086015 = (73.38857288474917 - 33.682080569830944) / 73.38857288474917 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Collections.Tests.DictionarySequentialKeys.ContainsKey_17_Int_32ByteValue ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 21.852883438786918 < 48.85490118323548. IsChangePoint: Marked as a change because one of 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 63.69963797777308 (T) = (0 -22.11762254483436) / Math.Sqrt((97.37608244186137 / (299)) + (0.3677435892027073 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.6269048255096833 = (59.28144896285277 - 22.11762254483436) / 59.28144896285277 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteRefsValue ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 33.460308236239456 < 67.97482483399924. IsChangePoint: Marked as a change because one of 3/18/2022 2:15:20 PM, 4/8/2022 2:40:20 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 23.964867393754236 (T) = (0 -33.24208998325153) / Math.Sqrt((587.1788059400485 / (299)) + (0.1632903773937479 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.5029687645995584 = (66.88128957623655 - 33.24208998325153) / 66.88128957623655 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteValue ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 22.839394722066256 < 63.539638498109376. IsChangePoint: Marked as a change because one of 3/21/2022 10:36:51 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 54.88586847808459 (T) = (0 -23.308293097725073) / Math.Sqrt((128.26584646137024 / (299)) + (0.5227531639380162 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.612319774281671 = (60.12247092184638 - 23.308293097725073) / 60.12247092184638 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.DictionarySequentialKeys.ContainsKey_3k_Int_32ByteValue ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 22.623679237543733 < 48.58181798871449. IsChangePoint: Marked as a change because one of 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 55.25457092607432 (T) = (0 -22.43884210359803) / Math.Sqrt((130.45436606935695 / (299)) + (0.36742346108563395 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.6231670640146241 = (59.54586226631962 - 22.43884210359803) / 59.54586226631962 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArraySetAll - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size%3a%204).html>) 69.66 ns 27.04 ns 0.39 0.63 False
[BitArraySetLengthShrink - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size%3a%20512).html>) 2.51 μs 524.37 ns 0.21 0.44 False
[BitArraySetLengthGrow - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.Perf_BitArray.BitArraySetLengthGrow(Size%3a%20512).html>) 3.28 μs 1.45 μs 0.44 0.47 False
[BitArrayCopyToByteArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size%3a%204).html>) 46.18 ns 16.15 ns 0.35 0.59 False
[BitArrayRightShift - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size%3a%204).html>) 40.81 ns 14.25 ns 0.35 0.32 False
[BitArraySetAll - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size%3a%20512).html>) 219.86 ns 54.10 ns 0.25 0.37 False
[BitArrayLeftShift - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size%3a%204).html>) 47.86 ns 11.95 ns 0.25 0.58 False
[BitArrayCopyToByteArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size%3a%20512).html>) 2.61 μs 266.92 ns 0.10 0.07 True
[BitArrayBoolArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.Perf_BitArray.BitArrayBoolArrayCtor(Size%3a%20512).html>) 49.35 μs 1.94 μs 0.04 0.42 False
[BitArrayByteArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size%3a%20512).html>) 2.48 μs 513.32 ns 0.21 0.43 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 27.040946494240277 < 34.557413694304245. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 3.4349066113517646 (T) = (0 -17.67002302291971) / Math.Sqrt((201.53144596229313 / (299)) + (29.73659625482389 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.2097129664387131 = (22.35899397626824 - 17.67002302291971) / 22.35899397626824 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 524.3701835137306 < 3.980072216307892. IsChangePoint: Marked as a change because one of 3/31/2022 1:23:38 AM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 61.515732570430906 (T) = (0 -651.2491501608768) / Math.Sqrt((862304.3846187851 / (299)) + (18602.942422526212 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.8505129588188389 = (4356.559237610686 - 651.2491501608768) / 4356.559237610686 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArraySetLengthGrow(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.4515715794361366 < 3.2740321700258157. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 65.51884774089582 (T) = (0 -1275.6531758117355) / Math.Sqrt((882563.4857490665 / (299)) + (14194.837973479827 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.7529043864083296 = (5162.589320261157 - 1275.6531758117355) / 5162.589320261157 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 16.15497692690698 < 44.354016669221096. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 34.64391706963261 (T) = (0 -18.458806369611136) / Math.Sqrt((508.9893722160976 / (299)) + (45.27279888849183 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.7786552132037124 = (83.3939061171543 - 18.458806369611136) / 83.3939061171543 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 14.25454332242186 < 35.93184090570454. IsChangePoint: Marked as a change because one of 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 17.834697253139733 (T) = (0 -17.569721335043152) / Math.Sqrt((13.388253686266339 / (299)) + (40.610167716050235 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.5673717914793063 = (40.61159441063757 - 17.569721335043152) / 40.61159441063757 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 54.10003959692962 < 170.73538978165706. IsChangePoint: Marked as a change because one of 4/9/2022 1:59:14 AM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 36.10668002734909 (T) = (0 -47.02791634671061) / Math.Sqrt((626.3018754264504 / (299)) + (23.42374834158907 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.5720653160189249 = (109.8950800370048 - 47.02791634671061) / 109.8950800370048 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 11.949717796295861 < 40.1010411921587. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/20/2022 12:55:12 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 8.431748140107043 (T) = (0 -15.401098944904444) / Math.Sqrt((81.50531621591706 / (299)) + (62.47958400647801 / (18))) is greater than 1.9675235316906419 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (18) - 2, .975) and 0.5143960037845012 = (31.71534638291944 - 15.401098944904444) / 31.71534638291944 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 266.92389277061517 < 3.5656313298724847. IsChangePoint: Marked as a change because one of 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 75.91922824448281 (T) = (0 -395.28815231937625) / Math.Sqrt((179355.3326829633 / (299)) + (37139.46492745558 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.8976534957061401 = (3862.2535771658095 - 395.28815231937625) / 3862.2535771658095 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayBoolArrayCtor(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.9409778724363032 < 47.715648311188424. IsChangePoint: Marked as a change because one of 4/8/2022 8:38:01 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 8.672220346711319 (T) = (0 -1948.115687614064) / Math.Sqrt((214414187.1822162 / (299)) + (1685.9549952949199 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.7903509746062946 = (9292.271614216408 - 1948.115687614064) / 9292.271614216408 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 513.3204791750961 < 3.7153143784959664. IsChangePoint: Marked as a change because one of 3/31/2022 9:26:55 AM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 64.18483685183854 (T) = (0 -621.6216898468487) / Math.Sqrt((828874.7799492269 / (299)) + (17704.49337595639 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.8589840551699147 = (4408.165974392905 - 621.6216898468487) / 4408.165974392905 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 2 years ago

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Decimal.ToString(value%3a%20123456.789).html>) 427.77 ns 282.78 ns 0.66 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Decimal*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Decimal.ToString(value: 123456.789) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 282.78166778723295 < 408.69713756977353. IsChangePoint: Marked as a change because one of 3/30/2022 8:03:41 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 61.23631109198852 (T) = (0 -287.1769220508568) / Math.Sqrt((2345.16474759316 / (267)) + (113.81979646688018 / (25))) is greater than 1.9681778964020553 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (267) + (25) - 2, .975) and 0.437794502373415 = (510.80418683774366 - 287.1769220508568) / 510.80418683774366 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.Buffers.Tests.NonStandardArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[RentNoReturn - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Buffers.Tests.NonStandardArrayPoolTests(Byte).RentNoReturn(RentalSize%3a%2064%2c%20UseSharedPool%3a%20True).html>) 183.37 ns 136.58 ns 0.74 0.36 False
[RentNoReturn - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Buffers.Tests.NonStandardArrayPoolTests(Byte).RentNoReturn(RentalSize%3a%2064%2c%20UseSharedPool%3a%20False).html>) 632.94 ns 325.87 ns 0.51 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.NonStandardArrayPoolTests&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.NonStandardArrayPoolTests<Byte>.RentNoReturn(RentalSize: 64, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 136.5817638888889 < 176.36758793589743. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/7/2022 6:53:49 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 12.244874507093146 (T) = (0 -139.01737627447875) / Math.Sqrt((1236.1029371245122 / (299)) + (27.90600179617499 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1679337056217223 = (167.07487998700023 - 139.01737627447875) / 167.07487998700023 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.NonStandardArrayPoolTests<Byte>.RentNoReturn(RentalSize: 64, UseSharedPool: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 325.87063966666665 < 504.04304018749997. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 27.56605962555695 (T) = (0 -326.25265936411705) / Math.Sqrt((3484.650190422965 / (299)) + (39.253499719079194 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.23504290607997982 = (426.4979852559264 - 326.25265936411705) / 426.4979852559264 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Transient - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/Microsoft.Extensions.DependencyInjection.GetService.Transient.html>) 5.38 μs 441.40 ns 0.08 0.41 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.DependencyInjection.GetService.Transient ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 441.4032533044197 < 5.011752109692813. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 9.234324341187264 (T) = (0 -452.21912470480265) / Math.Sqrt((4305138.051555526 / (299)) + (116.40155177679877 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.7102009798576167 = (1560.4577423437092 - 452.21912470480265) / 1560.4577423437092 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EncodeUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments%3a%20Url%2c%ef%bf%bd2020%2c16).html>) 339.98 ns 155.54 ns 0.46 0.56 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,�2020,16) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 155.53928976623803 < 214.56392137730018. IsChangePoint: Marked as a change because one of 3/30/2022 8:03:41 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 41.749093779198034 (T) = (0 -115.33447763903048) / Math.Sqrt((5500.81307894744 / (299)) + (440.12713798175747 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.684738569861416 = (365.8375767322099 - 115.33447763903048) / 365.8375767322099 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.Threading.Channels.Tests.BoundedChannelPerfTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryWriteThenTryRead - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Threading.Channels.Tests.BoundedChannelPerfTests.TryWriteThenTryRead.html>) 1.97 μs 426.21 ns 0.22 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Threading.Channels.Tests.BoundedChannelPerfTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Channels.Tests.BoundedChannelPerfTests.TryWriteThenTryRead ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 426.207936751552 < 1.8605372386489925. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 7.067425161026099 (T) = (0 -433.8426811260653) / Math.Sqrt((334170.2049809601 / (215)) + (56.59016835108951 / (25))) is greater than 1.9699815295298084 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (215) + (25) - 2, .975) and 0.3912468278987889 = (712.6742019734967 - 433.8426811260653) / 712.6742019734967 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LoadXml - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.LoadXml.html>) 29.43 μs 23.95 μs 0.81 0.38 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument*'
### Payloads [Baseline]() [Compare]() ### Histogram #### XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.LoadXml ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 23.954818627075355 < 27.95640468179281. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 12.706386938588064 (T) = (0 -25150.206927539413) / Math.Sqrt((1652812.224383011 / (299)) + (1082971.2502900271 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.10044443932843668 = (27958.48086221992 - 25150.206927539413) / 27958.48086221992 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.Memory.Constructors_ValueTypesOnly<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SpanFromPointerLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Memory.Constructors_ValueTypesOnly(Byte).SpanFromPointerLength.html>) 31.81 ns 1.15 ns 0.04 0.67 False
[ReadOnlyFromPointerLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Memory.Constructors_ValueTypesOnly(Byte).ReadOnlyFromPointerLength.html>) 31.90 ns 1.18 ns 0.04 0.44 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Constructors_ValueTypesOnly&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Constructors_ValueTypesOnly<Byte>.SpanFromPointerLength ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.1549169217932094 < 30.35692761696089. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 25.72192541937182 (T) = (0 -1.3748264223948348) / Math.Sqrt((83.177927757191 / (299)) + (0.30403745421760003 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.9097576714555311 = (15.234828761287533 - 1.3748264223948348) / 15.234828761287533 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Constructors_ValueTypesOnly<Byte>.ReadOnlyFromPointerLength ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.178070670599684 < 30.041010308624703. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 34.61640481931583 (T) = (0 -1.2512513534568448) / Math.Sqrt((54.581185062203275 / (299)) + (0.051584836488807566 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.9224010637539791 = (16.124594150232383 - 1.2512513534568448) / 16.124594150232383 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[HtmlDecode_Entities - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities.html>) 3.29 μs 658.85 ns 0.20 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Tests.Perf_WebUtility*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 658.8494812544163 < 3.1252083221936533. IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/10/2022 3:30:35 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 48.649646872961156 (T) = (0 -745.5221588759449) / Math.Sqrt((288732.08222815185 / (299)) + (15145.336909289696 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.7212044426652623 = (2674.0819186757303 - 745.5221588759449) / 2674.0819186757303 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.ScopeValidation

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Transient - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/Microsoft.Extensions.DependencyInjection.ScopeValidation.Transient.html>) 5.26 μs 456.42 ns 0.09 0.39 False
[TransientWithScopeValidation - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/Microsoft.Extensions.DependencyInjection.ScopeValidation.TransientWithScopeValidation.html>) 5.44 μs 582.48 ns 0.11 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.ScopeValidation*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.DependencyInjection.ScopeValidation.Transient ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 456.41947652611503 < 4.9970107070664. IsChangePoint: Marked as a change because one of 4/7/2022 6:53:49 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 5.663332625996439 (T) = (0 -454.43042302887943) / Math.Sqrt((2482249.7611120874 / (299)) + (143.5438368110952 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.5318143470475347 = (970.6201378943526 - 454.43042302887943) / 970.6201378943526 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.ScopeValidation.TransientWithScopeValidation ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 582.4775062446591 < 5.2086958252538205. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 9.689153110613104 (T) = (0 -577.5912760055436) / Math.Sqrt((4230184.141501719 / (299)) + (67.39769156894174 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.6661653410748205 = (1730.1716899772111 - 577.5912760055436) / 1730.1716899772111 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.Tests.Perf_Lazy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ValueFromAlreadyInitialized - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Lazy.ValueFromAlreadyInitialized.html>) 18.48 ns 13.11 ns 0.71 0.60 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Lazy*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Lazy.ValueFromAlreadyInitialized ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 13.113561064326097 < 17.558588129753144. IsChangePoint: Marked as a change because one of 3/22/2022 5:45:59 AM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 50.22055746304907 (T) = (0 -12.985380971584812) / Math.Sqrt((4.371889367156273 / (299)) + (0.16793554942036348 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.36100467368465167 = (20.321558604290075 - 12.985380971584812) / 20.321558604290075 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetByteCount - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Text.Tests.Perf_Encoding.GetByteCount(size%3a%2016%2c%20encName%3a%20%22utf-8%22).html>) 295.87 ns 67.80 ns 0.23 0.20 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "utf-8") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 67.8021999280736 < 144.01917205847317. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/8/2022 8:38:01 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 19.640295887782294 (T) = (0 -52.06156049065719) / Math.Sqrt((2146.70338948821 / (299)) + (60.46824880109953 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.5389090671531144 = (112.90952994719864 - 52.06156049065719) / 112.90952994719864 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.Tests.Perf_Environment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetFolderPath - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Environment.GetFolderPath(folder%3a%20System%2c%20option%3a%20None).html>) 2.91 μs 1.59 μs 0.55 0.46 False
[SetEnvironmentVariable - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.Tests.Perf_Environment.SetEnvironmentVariable.html>) 516.12 ns 397.66 ns 0.77 0.14 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Environment*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Environment.GetFolderPath(folder: System, option: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.587842722357967 < 2.771668434585059. IsChangePoint: Marked as a change because one of 1/5/2022 11:37:43 PM, 3/22/2022 12:26:23 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 6.3537570705641 (T) = (0 -1676.476568775632) / Math.Sqrt((296979.9024099568 / (299)) + (25477.95472303295 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1453097978088221 = (1961.502032523167 - 1676.476568775632) / 1961.502032523167 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Environment.SetEnvironmentVariable ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 397.6617383985441 < 570.2107723095497. IsChangePoint: Marked as a change because one of 3/22/2022 12:26:23 PM, 4/8/2022 8:38:01 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 22.523291208749733 (T) = (0 -404.77787199778584) / Math.Sqrt((2209.306004954596 / (299)) + (69.48832387294411 / (25))) is greater than 1.9673585853226652 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (25) - 2, .975) and 0.1506975699574141 = (476.6003930749255 - 404.77787199778584) / 476.6003930749255 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline f865423523c394cc6015e06f285442a6f909f3a5
Compare 4479fa685e6305db0b35a10793018c9a12203f8a
Diff Diff

Improvements in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WritePartialCharArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 18.04_LLVM=false_MonoAOT=true_MonoInterpreter=false/System.IO.Tests.Perf_StreamWriter.WritePartialCharArray(writeLength%3a%202).html>) 1.04 secs 657.98 ms 0.63 0.48 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_StreamWriter*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_StreamWriter.WritePartialCharArray(writeLength: 2) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 657.9823367333333 < 1.1342330480307143. IsChangePoint: Marked as a change because one of 4/9/2022 2:03:45 PM, 4/18/2022 5:19:34 PM, 4/25/2022 9:58:40 PM falls between 4/17/2022 3:50:42 AM and 4/25/2022 9:58:40 PM. IsImprovementStdDev: Marked as improvement because 45.674677280276384 (T) = (0 -516768076.60803664) / Math.Sqrt((46686536510120410 / (213)) + (8950458467190975 / (25))) is greater than 1.9700668531019387 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (213) + (25) - 2, .975) and 0.6798424143116183 = (1614105364.696938 - 516768076.60803664) / 1614105364.696938 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)