search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/arm64: 29 Improvements on 7/26/2023 2:29:36 PM #20370

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 8aff8937fef4d5db8742a2710371aae366813c44
Compare 08acbb7ad7154e9d18227e4b00e4c9f41af1fffa
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrueComparer<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
290.57 μs 267.82 μs 0.92 0.03 False
53.77 μs 50.47 μs 0.94 0.14 False
281.43 μs 258.72 μs 0.92 0.01 False
34.73 μs 32.15 μs 0.93 0.00 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrueComparer<String>.ImmutableSortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 267.81677966101694 < 276.6958745453042. IsChangePoint: Marked as a change because one of 5/14/2023 7:30:23 PM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 36.21724856131776 (T) = (0 -267682.1036098478) / Math.Sqrt((384097002.3553029 / (299)) + (2229692.955084129 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.13900924331805692 = (310900.0898469944 - 267682.1036098478) / 310900.0898469944 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<String>.ImmutableHashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 50.469218917470535 < 51.08812113805056. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/26/2023 10:32:59 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 34.23118615572104 (T) = (0 -50281.1361634821) / Math.Sqrt((6974457.710679948 / (299)) + (50032.794803228375 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.10044294375309443 = (55895.438554240194 - 50281.1361634821) / 55895.438554240194 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<String>.SortedSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 258.7235340479194 < 266.41780840416095. IsChangePoint: Marked as a change because one of 5/15/2023 9:18:21 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 32.47661128518794 (T) = (0 -259346.72362828485) / Math.Sqrt((178032454.98281887 / (299)) + (9242413.383413365 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.11943088976313242 = (294521.7139839509 - 259346.72362828485) / 294521.7139839509 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrueComparer<String>.HashSet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 32.14734975137174 < 33.088064154552974. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/19/2023 4:22:12 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 45.2325680006759 (T) = (0 -32126.7666393224) / Math.Sqrt((491526.748298015 / (299)) + (60243.5690901407 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.09383307539857726 = (35453.47525617683 - 32126.7666393224) / 35453.47525617683 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 8aff8937fef4d5db8742a2710371aae366813c44
Compare 08acbb7ad7154e9d18227e4b00e4c9f41af1fffa
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
16.51 μs 14.13 μs 0.86 0.29 False
14.98 μs 13.46 μs 0.90 0.34 False
15.39 μs 12.30 μs 0.80 0.30 False
15.62 μs 12.43 μs 0.80 0.35 False
17.79 μs 13.94 μs 0.78 0.32 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.128959084761048 < 15.82215129196296. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 30.737868207301688 (T) = (0 -13897.664672451774) / Math.Sqrt((3705707.597040337 / (299)) + (229157.40923912157 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.2655201979381412 = (18921.779242176213 - 13897.664672451774) / 18921.779242176213 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 13.455286250766404 < 14.312317188759422. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 30.069383567310666 (T) = (0 -12115.477468941068) / Math.Sqrt((3303338.731003966 / (299)) + (261919.35171087546 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.2912634714771685 = (17094.473025388554 - 12115.477468941068) / 17094.473025388554 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToStream(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.302770702716824 < 14.610578926433332. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 23.928517298469178 (T) = (0 -12721.917553768493) / Math.Sqrt((3132797.1378117446 / (299)) + (454623.5586603488 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.2705676571089315 = (17440.846540127095 - 12721.917553768493) / 17440.846540127095 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 12.433099376794768 < 14.815954269948799. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 33.28992405959314 (T) = (0 -12704.993708036436) / Math.Sqrt((3543609.6077895546 / (299)) + (177206.31284370448 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.28404963408618156 = (17745.634771511217 - 12704.993708036436) / 17745.634771511217 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 37.1819272579127 (T) = (0 -13969.850289721508) / Math.Sqrt((4957315.684918725 / (299)) + (90414.96950473945 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.28409847144320166 = (19513.64780276924 - 13969.850289721508) / 19513.64780276924 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 8aff8937fef4d5db8742a2710371aae366813c44
Compare 08acbb7ad7154e9d18227e4b00e4c9f41af1fffa
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
324.12 ms 244.51 ms 0.75 0.01 False
327.03 ms 244.76 ms 0.75 0.01 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'
### Payloads [Baseline]() [Compare]() ### LinqBenchmarks.Where01LinqMethodX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 115.46590170274267 (T) = (0 -244922381.41483513) / Math.Sqrt((181624193362364.5 / (299)) + (1527003582355.8127 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.28327925140425037 = (341726372.3628826 - 244922381.41483513) / 341726372.3628826 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### LinqBenchmarks.Where01LinqQueryX #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 122.6850137901577 (T) = (0 -244127500.9775641) / Math.Sqrt((177775700929005.44 / (299)) + (632137304789.873 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.28580147518008675 = (341820225.74062496 - 244127500.9775641) / 341820225.74062496 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 8aff8937fef4d5db8742a2710371aae366813c44
Compare 08acbb7ad7154e9d18227e4b00e4c9f41af1fffa
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
26.95 μs 20.65 μs 0.77 0.51 False
25.73 μs 22.18 μs 0.86 0.52 False
25.77 μs 18.63 μs 0.72 0.57 False
21.92 μs 17.31 μs 0.79 0.57 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 23.329226337583027 (T) = (0 -20246.48342971172) / Math.Sqrt((5605503.06268346 / (299)) + (1703002.155229112 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.289614928520781 = (28500.716361554332 - 20246.48342971172) / 28500.716361554332 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 22.178358001616488 < 24.023687244921074. IsChangePoint: Marked as a change because one of 5/19/2023 1:23:34 PM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 12.449432395109888 (T) = (0 -20163.77565974517) / Math.Sqrt((5865628.032640092 / (299)) + (2676934.7129812306 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.21069617339085553 = (25546.27885990229 - 20163.77565974517) / 25546.27885990229 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 18.633264007698887 < 24.665278203660908. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/26/2023 2:29:36 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 13.145213711381572 (T) = (0 -20522.092040610223) / Math.Sqrt((6253621.212131092 / (299)) + (3884906.6683789003 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.2531107461002623 = (27476.75365987406 - 20522.092040610223) / 27476.75365987406 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToStream(Mode: Reflection) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.308069096779338 < 23.01473853872619. IsChangePoint: Marked as a change because one of 5/18/2023 11:09:10 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 13.132048205679633 (T) = (0 -20084.43838149829) / Math.Sqrt((5394097.503282559 / (299)) + (3255296.954464726 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.23531052055170892 = (26264.828955131998 - 20084.43838149829) / 26264.828955131998 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 8aff8937fef4d5db8742a2710371aae366813c44
Compare 08acbb7ad7154e9d18227e4b00e4c9f41af1fffa
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
990.95 μs 902.87 μs 0.91 0.22 False
1.08 ms 903.26 μs 0.84 0.27 False
1.08 ms 906.37 μs 0.84 0.24 False
1.08 ms 903.41 μs 0.84 0.23 False
1.00 ms 913.23 μs 0.91 0.22 False
1.58 ms 592.35 μs 0.37 0.03 True

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsFalse<String>.Stack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 902.873363095238 < 941.9219282816791. IsChangePoint: Marked as a change because one of 5/19/2023 9:46:56 PM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 69.58666972996511 (T) = (0 -903086.2701751372) / Math.Sqrt((1487959234.4585483 / (299)) + (1271286.2574554149 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.14767344314393388 = (1059554.3021753374 - 903086.2701751372) / 1059554.3021753374 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsFalse<String>.List(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 903.2595085470086 < 0.9900280556175594. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/6/2023 1:55:21 PM, 5/20/2023 7:49:01 PM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 50.69346764108379 (T) = (0 -903052.5831807081) / Math.Sqrt((2471934149.600149 / (299)) + (1242792.9467629124 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.1395356962712419 = (1049494.5336690864 - 903052.5831807081) / 1049494.5336690864 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsFalse<String>.Queue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 906.3657986111111 < 1.0242627268295939. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/6/2023 1:55:21 PM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 53.669627083274534 (T) = (0 -906622.9101681738) / Math.Sqrt((2333518261.2757263 / (299)) + (401183.7863697599 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.14210309274578384 = (1056797.0376183195 - 906622.9101681738) / 1056797.0376183195 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsFalse<String>.ICollection(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 903.4094328703704 < 1.0191966504693224. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/6/2023 1:55:21 PM, 5/19/2023 1:23:34 PM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 53.88585850130679 (T) = (0 -903092.5110414759) / Math.Sqrt((2026283561.4830449 / (299)) + (1192202.0205824925 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.1350844399305114 = (1044139.5122651281 - 903092.5110414759) / 1044139.5122651281 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsFalse<String>.ImmutableArray(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 913.226469017094 < 1.0271254389880953. IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 4/6/2023 1:55:21 PM, 5/21/2023 3:58:09 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 59.024630915990585 (T) = (0 -911310.956260175) / Math.Sqrt((1666502299.4354825 / (299)) + (2059825.8164444517 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.13394811276572993 = (1052259.0732645816 - 911310.956260175) / 1052259.0732645816 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsFalse<String>.LinkedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 592.350284900285 < 1.4127751096299535. IsChangePoint: Marked as a change because one of 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 433.12566188442855 (T) = (0 -590471.6822286028) / Math.Sqrt((1301210698.4444215 / (299)) + (2014909.5737530699 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.6081810957837187 = (1507001.5149209506 - 590471.6822286028) / 1507001.5149209506 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 8aff8937fef4d5db8742a2710371aae366813c44
Compare 08acbb7ad7154e9d18227e4b00e4c9f41af1fffa
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
591.59 μs 464.11 μs 0.78 0.25 False
583.54 μs 458.22 μs 0.79 0.21 False
586.56 μs 457.29 μs 0.78 0.23 False
762.86 μs 329.23 μs 0.43 0.07 True
544.73 μs 458.18 μs 0.84 0.25 False
583.41 μs 457.93 μs 0.78 0.23 False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrue<String>.ImmutableArray(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 464.10505514705886 < 541.4943159893269. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 58.97247013500794 (T) = (0 -463733.12649819546) / Math.Sqrt((718610578.9126096 / (299)) + (118646.68547934495 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.16489340249482523 = (555298.1234773707 - 463733.12649819546) / 555298.1234773707 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrue<String>.Queue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 458.22487637362633 < 515.2075191143268. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/6/2023 1:55:21 PM, 5/21/2023 4:09:07 PM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 58.11261208918098 (T) = (0 -458280.908698816) / Math.Sqrt((698594136.0321223 / (299)) + (71826.54699415927 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.16248864877519895 = (547193.668514 - 458280.908698816) / 547193.668514 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrue<String>.ICollection(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 457.2868239795919 < 556.040551252374. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/6/2023 5:23:20 AM, 5/19/2023 1:23:34 PM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 59.88053282098031 (T) = (0 -457446.40417729603) / Math.Sqrt((682662527.4669964 / (299)) + (129081.3330792023 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.16537511171195654 = (548086.2248375984 - 457446.40417729603) / 548086.2248375984 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrue<String>.LinkedList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 329.2326475694444 < 722.5636707057822. IsChangePoint: Marked as a change because one of 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 354.284697742803 (T) = (0 -327571.2525144397) / Math.Sqrt((426365956.3483358 / (299)) + (572630.3439474889 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.5666550483593557 = (755913.3924931043 - 327571.2525144397) / 755913.3924931043 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrue<String>.Stack(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 458.1814923469388 < 517.5840436126373. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 92.62751744536219 (T) = (0 -459195.88088805234) / Math.Sqrt((307252962.1939079 / (299)) + (354358.3892835785 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.1712754240834175 = (554099.5093335737 - 459195.88088805234) / 554099.5093335737 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.ContainsTrue<String>.List(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 457.92578296703294 < 554.0260633680554. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 57.37371521291157 (T) = (0 -457703.0321657509) / Math.Sqrt((754797070.1168795 / (299)) + (370046.5880608888 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.1667175481057661 = (549277.1762147294 - 457703.0321657509) / 549277.1762147294 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 8aff8937fef4d5db8742a2710371aae366813c44
Compare 08acbb7ad7154e9d18227e4b00e4c9f41af1fffa
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
37.13 ns 30.90 ns 0.83 0.01 True
72.14 ns 65.21 ns 0.90 0.02 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 54.5057681319457 (T) = (0 -29.92593792210394) / Math.Sqrt((0.23362591205550345 / (299)) + (0.19886969789293799 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.17310453440382534 = (36.19071474836055 - 29.92593792210394) / 36.19071474836055 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking. IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 7/26/2023 12:02:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM. IsImprovementStdDev: Marked as improvement because 40.80348345315066 (T) = (0 -65.34473107573866) / Math.Sqrt((1.550784629493671 / (299)) + (0.2471617391069444 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.08231411592322224 = (71.20598911846358 - 65.34473107573866) / 71.20598911846358 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)