search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Windows/x64: 17 Regressions on 9/19/2023 3:42:14 PM #22210

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Perf_Frozen<ReferenceType>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
56.05 ns 59.66 ns 1.06 0.02 True
56.18 ns 61.51 ns 1.09 0.01 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_Frozen&lt;ReferenceType&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Perf_Frozen<ReferenceType>.Contains_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.660194562155354 > 58.85155954141871. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -39.973991245103 (T) = (0 -59.44105410848719) / Math.Sqrt((0.07804152328652428 / (21)) + (0.07442789115354381 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.05879108503690587 = (56.140493576610794 - 59.44105410848719) / 56.140493576610794 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Perf_Frozen<ReferenceType>.TryGetValue_True(Count: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 61.50871989267825 > 58.58186623634821. IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -74.91402401413302 (T) = (0 -61.635412451289476) / Math.Sqrt((0.07289595413694154 / (21)) + (0.05464703263555191 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.10150438876356882 = (55.95566670458282 - 61.635412451289476) / 55.95566670458282 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Xml.Linq.Perf_XElementList

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
192.77 ns 206.79 ns 1.07 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElementList*'
### Payloads [Baseline]() [Compare]() ### System.Xml.Linq.Perf_XElementList.Enumerator #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 206.78810330684536 > 200.71327445116515. IsChangePoint: Marked as a change because one of 7/13/2023 8:36:41 PM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -20.7666714769401 (T) = (0 -204.70085113905358) / Math.Sqrt((3.3897205535240973 / (21)) + (4.935144573817205 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.06548711602298993 = (192.11949920437777 - 204.70085113905358) / 192.11949920437777 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.16 μs 19.58 μs 1.14 0.18 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrue<Int32>.ICollection(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.57536973299958 > 18.02302663118132. IsChangePoint: Marked as a change because one of 7/17/2023 6:26:12 PM, 7/23/2023 4:13:29 AM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -172.2532715484554 (T) = (0 -19590.213796033928) / Math.Sqrt((880.2399900085569 / (21)) + (3743.7135245588956 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.1411565721757706 = (17166.981528820812 - 19590.213796033928) / 17166.981528820812 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
379.14 μs 416.60 μs 1.10 0.21 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.ContainsTrue<String>.List(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 416.6028728070176 > 397.78310937500004. IsChangePoint: Marked as a change because one of 7/26/2023 12:02:55 AM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -10.756097635030265 (T) = (0 -433151.0171463505) / Math.Sqrt((2258675.010288579 / (21)) + (609798851.7340239 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.14342619683041283 = (378818.5178431707 - 433151.0171463505) / 378818.5178431707 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
162.14 μs 187.10 μs 1.15 0.02 False
372.70 μs 404.08 μs 1.08 0.04 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 187.0991865079365 > 170.19959171788264. IsChangePoint: Marked as a change because one of 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -67.80046181021837 (T) = (0 -187205.07007811137) / Math.Sqrt((2955478.1907850965 / (21)) + (54770.23796687053 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.15870263443727656 = (161564.3777050938 - 187205.07007811137) / 161564.3777050938 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodLongHierarchy #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 404.0755235042736 > 391.2438697916666. IsChangePoint: Marked as a change because one of 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 7/17/2023 6:26:12 PM, 7/23/2023 4:13:29 AM, 8/8/2023 9:37:01 AM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -31.004844284895416 (T) = (0 -403127.3032518697) / Math.Sqrt((8842985.12446827 / (21)) + (14310815.957338242 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.08410041432465892 = (371854.2101129978 - 403127.3032518697) / 371854.2101129978 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.09 μs 1.19 μs 1.09 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.TryGetValueFalse<Int32, Int32>.ConcurrentDictionary(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1889053520114943 > 1.148172145162915. IsChangePoint: Marked as a change because one of 6/16/2023 3:00:20 AM, 7/27/2023 4:03:50 PM, 8/9/2023 9:46:11 AM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -21.67831254303004 (T) = (0 -1176.2603530743036) / Math.Sqrt((7.477471787860056 / (21)) + (337.4298904709855 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.07523920591898872 = (1093.9522541581562 - 1176.2603530743036) / 1093.9522541581562 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.79 μs 3.00 μs 1.08 0.32 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.AddGivenSize<String>.ICollection(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.002444875377401 > 2.9208132992023743. IsChangePoint: Marked as a change because one of 9/2/2023 9:37:00 AM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -16.508612758143514 (T) = (0 -3024.232159443172) / Math.Sqrt((713.5394635087175 / (21)) + (4459.053487726782 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.08805024082634125 = (2779.496797083891 - 3024.232159443172) / 2779.496797083891 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.40 ns 6.88 ns 1.27 0.08 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'
### Payloads [Baseline]() [Compare]() ### System.Text.Perf_Ascii.Equals_Bytes(Size: 6) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.882139765501554 > 5.662227208608046. IsChangePoint: Marked as a change because one of 7/14/2023 2:26:40 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -503.3583494305262 (T) = (0 -6.865547782043606) / Math.Sqrt((8.890247511383571E-05 / (21)) + (0.0001054810084822758 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.27447346223195673 = (5.386968018949666 - 6.865547782043606) / 5.386968018949666 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.BenchE

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
300.47 ms 347.32 ms 1.16 0.04 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BenchE*'
### Payloads [Baseline]() [Compare]() ### Benchstone.BenchI.BenchE.Test #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 347.3235833333333 > 315.67632375000005. IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 2:12:47 AM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -338.95242800512324 (T) = (0 -346439795.6707875) / Math.Sqrt((225023890097.62817 / (21)) + (173799340991.18283 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.1508907921669502 = (301018826.4852608 - 346439795.6707875) / 301018826.4852608 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
848.25 ns 897.39 ns 1.06 0.16 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Reader*'
### Payloads [Baseline]() [Compare]() ### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: LotsOfStrings) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 897.3934086194976 > 893.2924201835257. IsChangePoint: Marked as a change because one of 6/2/2023 6:53:36 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -5.312389724657342 (T) = (0 -918.0001117048805) / Math.Sqrt((1148.3036128769281 / (21)) + (1575.9034342043597 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.0677865351332506 = (859.722502110707 - 918.0001117048805) / 859.722502110707 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEachNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.62 μs 2.92 μs 1.11 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateForEachNonGeneric<Int32>.Queue(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9171172682477367 > 2.7476413247067017. IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 7/5/2023 7:07:19 PM, 7/23/2023 4:13:29 AM, 8/10/2023 8:47:51 AM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -259.0129732197269 (T) = (0 -2909.080949309966) / Math.Sqrt((6.81633621806005 / (21)) + (22.54377736262768 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.11123172564140371 = (2617.8886745074187 - 2909.080949309966) / 2617.8886745074187 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
812.30 ns 891.85 ns 1.10 0.06 False
861.24 ns 969.72 ns 1.13 0.07 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 891.8531775576047 > 853.2318270657577. IsChangePoint: Marked as a change because one of 7/5/2023 7:07:19 PM, 7/12/2023 9:48:15 PM, 8/14/2023 10:44:26 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -241.53110973155404 (T) = (0 -890.1687121611484) / Math.Sqrt((0.5483375566962242 / (21)) + (1.8450520421332794 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.09538061725612089 = (812.6569871128274 - 890.1687121611484) / 812.6569871128274 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 969.7214441332088 > 906.4495768605274. IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 7/5/2023 7:07:19 PM, 7/12/2023 9:48:15 PM, 8/14/2023 10:44:26 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -115.9918655530448 (T) = (0 -967.3376720241808) / Math.Sqrt((14.693605750929631 / (21)) + (2.770940288344323 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.12140326803239056 = (862.613566055918 - 967.3376720241808) / 862.613566055918 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.25 μs 2.42 μs 1.07 0.07 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateFor<String>.IList(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4178446740981583 > 2.365889536141971. IsChangePoint: Marked as a change because one of 6/5/2023 10:43:22 AM, 6/19/2023 6:37:24 PM, 7/1/2023 6:36:20 PM, 7/12/2023 2:12:47 AM, 8/15/2023 7:48:35 PM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -209.62537206256792 (T) = (0 -2412.578534746531) / Math.Sqrt((4.110388072354371 / (21)) + (9.029960461041224 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.07033525453184473 = (2254.0400538350686 - 2412.578534746531) / 2254.0400538350686 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19445f05aae8a04a2d436e42908429cd90ac1ab8
Compare 1185d19592d2a3f2ae79bd6cb0d7503db09b7d1d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
42.48 ns 49.50 ns 1.17 0.26 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<UInt16>.GetHashCodeBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 49.50368678843403 > 44.75154712631811. IsChangePoint: Marked as a change because one of 6/30/2023 8:34:57 AM, 7/14/2023 2:26:40 PM, 8/14/2023 10:44:26 PM, 8/28/2023 1:23:07 PM, 9/19/2023 8:08:41 AM, 9/25/2023 11:21:57 AM falls between 9/16/2023 7:32:07 PM and 9/25/2023 11:21:57 AM. IsRegressionStdDev: Marked as regression because -16.913296959294154 (T) = (0 -50.05626980435887) / Math.Sqrt((2.9046753782401504 / (21)) + (0.9479575921219865 / (24))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (24) - 2, .025) and -0.166154199420524 = (42.92422891349397 - 50.05626980435887) / 42.92422891349397 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
cincuranet commented 1 year ago

The diff for Perf_Frozen points to https://github.com/dotnet/runtime/compare/e4df4329df8e78c21530ae1f4af32dd87e950825...0eb038e3bdb417dd3f97f64134f5d259cc246711. Nothing pops up there. And overall it's ~3-4ns. Is it worth transferring @AndyAyersMS?

Rest is noise.