search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/arm64: 37 Regressions on 7/18/2023 9:59:03 AM #20085

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
560.52 ns 1.77 μs 3.16 0.32 True
211.10 ns 440.67 ns 2.09 0.06 True
185.46 ns 433.14 ns 2.34 0.08 True
988.54 ns 1.83 μs 1.86 0.27 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 10000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.772373048963904 > 597.7723705645614. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -60.60426905675094 (T) = (0 -1827.9988778594964) / Math.Sqrt((266.79985715528363 / (7)) + (4288.520717203728 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -2.1833492176703917 = (574.2376198352644 - 1827.9988778594964) / 574.2376198352644 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Byte>.AllocateArray(length: 1000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 440.6663579764139 > 221.63016614943743. IsChangePoint: Marked as a change because one of 5/23/2023 1:41:12 PM, 6/8/2023 5:31:08 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -98.31658260006557 (T) = (0 -444.6414966261009) / Math.Sqrt((18.490886883709933 / (7)) + (32.81613348544553 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -1.102656302973108 = (211.46656065348768 - 444.6414966261009) / 211.46656065348768 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 1000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 433.13923412480597 > 192.09062136893053. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -46.38473523317997 (T) = (0 -446.39772456274653) / Math.Sqrt((28.284107825832518 / (7)) + (305.4735695486712 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -1.415801845278139 = (184.78242552685498 - 446.39772456274653) / 184.78242552685498 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Byte>.AllocateArray(length: 10000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8342940949354902 > 1032.5791835568489. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -114.84185403501822 (T) = (0 -1851.8359929350418) / Math.Sqrt((223.9596749018958 / (7)) + (302.2893703359827 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -0.8831088532642023 = (983.3929619761748 - 1851.8359929350418) / 983.3929619761748 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
147.79 μs 187.64 μs 1.27 0.01 True
265.34 μs 298.40 μs 1.12 0.02 False
310.36 μs 348.52 μs 1.12 0.01 True
197.44 μs 228.31 μs 1.16 0.02 False
163.96 μs 203.67 μs 1.24 0.02 False
199.20 μs 228.56 μs 1.15 0.02 False
37.10 μs 72.71 μs 1.96 0.16 True
135.23 μs 158.90 μs 1.18 0.04 False
52.92 μs 124.31 μs 2.35 0.03 True
164.15 μs 234.37 μs 1.43 0.01 True

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'
### Payloads [Baseline]() [Compare]() ### System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishMostlyAscii) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 187.63716775412087 > 155.5525069973467. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -80.35305995285024 (T) = (0 -187070.5000325036) / Math.Sqrt((215205.5561740418 / (7)) + (2476608.046200222 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -0.2644856555018095 = (147941.97088638772 - 187070.5000325036) / 147941.97088638772 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: Chinese) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 298.3969203555878 > 277.9964584375. IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -68.70528361165759 (T) = (0 -299279.7666611301) / Math.Sqrt((318132.97128776275 / (7)) + (2448592.5045643346 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -0.12952083665564135 = (264961.70495380776 - 299279.7666611301) / 264961.70495380776 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: Greek) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 348.5158205555556 > 326.29254498468134. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -98.5361119781207 (T) = (0 -347889.3972830851) / Math.Sqrt((393884.7402266772 / (7)) + (1086080.5723560906 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -0.11845333283194574 = (311045.07186028257 - 347889.3972830851) / 311045.07186028257 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 228.31481606280192 > 207.029263447142. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/22/2023 12:06:43 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -92.97601675339924 (T) = (0 -226529.8209958274) / Math.Sqrt((163699.79200999573 / (7)) + (905425.4632060095 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -0.14815159729343813 = (197299.5739672626 - 226529.8209958274) / 197299.5739672626 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 203.6685387987013 > 172.17774774251302. IsChangePoint: Marked as a change because one of 5/6/2023 11:12:53 AM, 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -93.1642651702836 (T) = (0 -204754.73461489117) / Math.Sqrt((72742.96294049318 / (7)) + (1998669.5648389645 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.24909300375207058 = (163922.72953242194 - 204754.73461489117) / 163922.72953242194 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: Greek) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 228.56306642512078 > 209.15642305181964. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -86.67576979033494 (T) = (0 -228659.74598474192) / Math.Sqrt((109081.21806376819 / (7)) + (1188294.055964618 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -0.14721785115015298 = (199316.76076649013 - 228659.74598474192) / 199316.76076649013 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: EnglishAllAscii) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 72.71335439814817 > 39.09016951346855. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -47.27661693516247 (T) = (0 -74693.66670195) / Math.Sqrt((543985.6255477708 / (7)) + (5878150.891244419 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.980935456322766 = (37706.25966814928 - 74693.66670195) / 37706.25966814928 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 158.90002247474746 > 142.2134049639423. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -112.42964178634139 (T) = (0 -158911.17477942997) / Math.Sqrt((102627.73827593029 / (7)) + (374141.4229739779 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -0.1729669584548879 = (135477.9634958845 - 158911.17477942997) / 135477.9634958845 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: EnglishAllAscii) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 124.30534778164748 > 52.637405132088496. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -123.99822529393975 (T) = (0 -123803.44023559015) / Math.Sqrt((1400468.4257020904 / (7)) + (1922145.8292874435 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -1.443668020894427 = (50662.95387794772 - 123803.44023559015) / 50662.95387794772 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Utf8Encoding.GetString(Input: EnglishMostlyAscii) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 234.371946228678 > 172.20709305889423. IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -84.50647389114388 (T) = (0 -238014.76211736028) / Math.Sqrt((205982.66184004396 / (7)) + (9570681.374568477 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -0.45067934348160954 = (164071.24233679945 - 238014.76211736028) / 164071.24233679945 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
32.51 ns 34.76 ns 1.07 0.04 False
19.15 ns 20.21 ns 1.06 0.05 False
37.99 ns 40.83 ns 1.07 0.05 False
24.06 ns 26.15 ns 1.09 0.05 False
18.65 ns 20.21 ns 1.08 0.07 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 34.759155481450776 > 34.22410442875136. IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 5/3/2023 7:05:50 AM, 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -16.576215276380665 (T) = (0 -34.916324532556885) / Math.Sqrt((0.014873020465611004 / (7)) + (0.18403604481407315 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.06973475907621417 = (32.6401701321825 - 34.916324532556885) / 32.6401701321825 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.207191357248437 > 20.08250030482864. IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -19.769016222426178 (T) = (0 -20.130031089538996) / Math.Sqrt((0.010058750483299702 / (7)) + (0.014329739311865253 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.05418859468960922 = (19.095284459481366 - 20.130031089538996) / 19.095284459481366 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 40.82595064607403 > 39.711708135492415. IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -9.810962893047416 (T) = (0 -40.43031821659579) / Math.Sqrt((0.017871291749938654 / (7)) + (0.7456780138104244 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.06878640332441104 = (37.82824902229215 - 40.43031821659579) / 37.82824902229215 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.153133837228985 > 25.201664824704935. IsChangePoint: Marked as a change because one of 4/7/2023 2:04:00 PM, 4/17/2023 5:51:51 AM, 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -48.78200506367172 (T) = (0 -26.1247243418791) / Math.Sqrt((0.004491263756311857 / (7)) + (0.012508441987910452 / (10))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (10) - 2, .025) and -0.08841258119991163 = (24.002593127946124 - 26.1247243418791) / 24.002593127946124 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.208914950383658 > 19.788266771383235. IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -22.134313483073186 (T) = (0 -20.18565304864218) / Math.Sqrt((0.016123788171169082 / (7)) + (0.019742333797039067 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.07549643914780767 = (18.768684222364055 - 20.18565304864218) / 18.768684222364055 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.63 ns 3.66 ns 1.39 0.54 False
5.85 ns 7.67 ns 1.31 0.45 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'
### Payloads [Baseline]() [Compare]() ### System.Text.Perf_Ascii.IsValid_Bytes(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.6583753997813293 > 2.761317668143437. IsChangePoint: Marked as a change because one of 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -11.54626716894506 (T) = (0 -3.5320893775600735) / Math.Sqrt((0.008957991921871765 / (7)) + (0.05354568619031552 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -0.32926699353765726 = (2.6571707525513095 - 3.5320893775600735) / 2.6571707525513095 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Perf_Ascii.IsValid_Chars(Size: 128) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.673571111051339 > 6.366361358947512. IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -4.969352155265449 (T) = (0 -7.651018748561356) / Math.Sqrt((1.1067354089459571 / (7)) + (0.02458003130191897 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.35149085099051197 = (5.661169472922365 - 7.651018748561356) / 5.661169472922365 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
204.53 μs 399.02 μs 1.95 0.35 True
360.56 μs 598.14 μs 1.66 0.58 True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'
### Payloads [Baseline]() [Compare]() ### System.IO.Tests.Perf_File.ReadAllBytes(size: 1048576) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 399.0226697115385 > 201.73086745371134. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -34.888432585724175 (T) = (0 -386036.1312777556) / Math.Sqrt((126905722.66457008 / (7)) + (136836786.05678132 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.9987338745720759 = (193140.3355838981 - 386036.1312777556) / 193140.3355838981 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.IO.Tests.Perf_File.ReadAllBytesAsync(size: 1048576) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 598.1381092592592 > 356.4032068347617. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -23.555660524544834 (T) = (0 -595683.8856874101) / Math.Sqrt((411168358.77770424 / (7)) + (631129501.4412208 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -0.7159847314654284 = (347138.2202676733 - 595683.8856874101) / 347138.2202676733 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_GC<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
311.08 ns 762.20 ns 2.45 0.13 True
980.40 ns 2.50 μs 2.55 0.37 True
249.81 ns 786.64 ns 3.15 0.14 True
1.80 μs 2.57 μs 1.43 0.21 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_GC<Char>.AllocateArray(length: 1000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 762.1989856437098 > 328.13708511436096. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -73.51854524326161 (T) = (0 -772.1437049560932) / Math.Sqrt((79.94763897682792 / (7)) + (353.82841677147644 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -1.450064543736331 = (315.15239340535067 - 772.1437049560932) / 315.15239340535067 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 10000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.502489836130803 > 1.086687405469963. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -81.95726245316374 (T) = (0 -2550.7462633822997) / Math.Sqrt((607.0589793260359 / (7)) + (2822.8039843087176 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -1.4713620744184088 = (1032.121634375478 - 2550.7462633822997) / 1032.121634375478 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 1000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 786.6436786096733 > 262.63602575017745. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -150.49961777787698 (T) = (0 -767.9385296204238) / Math.Sqrt((18.638240401061115 / (7)) + (110.39172705794248 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -2.076666992342937 = (249.60079577400896 - 767.9385296204238) / 249.60079577400896 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_GC<Char>.AllocateArray(length: 10000, pinned: True) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.5699842798311185 > 1.8844891539655866. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -28.962995202086052 (T) = (0 -2628.497687405961) / Math.Sqrt((1310.1339031338375 / (7)) + (7712.67397680876 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -0.4650462048245016 = (1794.1397880490942 - 2628.497687405961) / 1794.1397880490942 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Tests.Perf_Interlocked

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.71 ns 8.29 ns 1.45 0.68 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Interlocked*'
### Payloads [Baseline]() [Compare]() ### System.Threading.Tests.Perf_Interlocked.CompareExchange_object_NoMatch #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.287434722697482 > 6.074797652207015. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -3.006763806424823 (T) = (0 -7.296404517441258) / Math.Sqrt((0.4963916482408503 / (7)) + (1.025651428802869 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.20042469997200466 = (6.078185926706955 - 7.296404517441258) / 6.078185926706955 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Hashing

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.14 ns 5.98 ns 1.44 0.15 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Hashing*'
### Payloads [Baseline]() [Compare]() ### System.Hashing.GetStringHashCode(BytesCount: 10) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.981298897751179 > 4.5566543281535035. IsChangePoint: Marked as a change because one of 3/3/2023 4:19:26 AM, 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -4.21690251383709 (T) = (0 -5.628034778155371) / Math.Sqrt((0.006178867849578617 / (7)) + (1.0603286526530222 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.3049548948333118 = (4.312819393557865 - 5.628034778155371) / 4.312819393557865 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
21.44 μs 66.31 μs 3.09 0.22 True
26.08 μs 71.63 μs 2.75 0.12 True
10.44 μs 13.18 μs 1.26 0.24 True
17.83 μs 69.23 μs 3.88 0.19 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
### Payloads [Baseline]() [Compare]() ### System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length: 100000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 66.3124805497199 > 21.88108119244488. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -107.94282096910455 (T) = (0 -67617.51315216393) / Math.Sqrt((170698.15557288603 / (7)) + (2110346.847446524 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -2.223835472902021 = (20974.24441182671 - 67617.51315216393) / 20974.24441182671 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 71.62569462256494 > 27.09267678650596. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -107.19067049406463 (T) = (0 -71471.400999289) / Math.Sqrt((362596.7133208583 / (7)) + (1555227.5693186298 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -1.768330743218914 = (25817.50796011631 - 71471.400999289) / 25817.50796011631 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.179595359078588 > 10.775661766353341. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -20.73245256363623 (T) = (0 -13720.620599087446) / Math.Sqrt((49827.935327235224 / (7)) + (238417.4410026215 / (12))) is less than -2.109815577813699 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (12) - 2, .025) and -0.3301891264602047 = (10314.789322928604 - 13720.620599087446) / 10314.789322928604 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### System.Text.Tests.Perf_StringBuilder.ctor_string(length: 100000) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 69.22902264957266 > 18.796329378707906. IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -67.80888639025984 (T) = (0 -70460.2706908977) / Math.Sqrt((183980.92578802313 / (7)) + (6289186.477514069 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -2.9096475877555075 = (18022.15394338094 - 70460.2706908977) / 18022.15394338094 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.76 ns 11.05 ns 1.13 0.09 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'
### Payloads [Baseline]() [Compare]() ### System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "utf-8") #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.04554009302548 > 10.47548769798093. IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -3.0185442870592736 (T) = (0 -11.272455539242896) / Math.Sqrt((0.8097278303689728 / (7)) + (0.014871061229794763 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.10084358203508208 = (10.239833999308052 - 11.272455539242896) / 10.239833999308052 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
148.76 μs 170.83 μs 1.15 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### MicroBenchmarks.Serializers.Json_ToString<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 170.83232998188407 > 155.4106443352005. IsChangePoint: Marked as a change because one of 5/19/2023 1:23:34 PM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -46.94897717868101 (T) = (0 -171304.6519091149) / Math.Sqrt((805864.3246059899 / (7)) + (1735544.1434469242 / (13))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (13) - 2, .025) and -0.1582878208505702 = (147894.71910644806 - 171304.6519091149) / 147894.71910644806 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 2ee61bbe7bb04e97159f017c4f9eaf486dc46c1f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.81 ns 5.29 ns 1.39 0.42 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'
### Payloads [Baseline]() [Compare]() ### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Object_Valid #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.290758956869743 > 3.997002343024921. IsChangePoint: Marked as a change because one of 5/11/2023 10:01:37 AM, 6/1/2023 5:52:46 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -10.851906765574313 (T) = (0 -5.137779883633514) / Math.Sqrt((0.09345596494170001 / (7)) + (0.026405725803738546 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.36070678717725946 = (3.775817047471091 - 5.137779883633514) / 3.775817047471091 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline bb38848e9877dc0b43860dc219ebe36fc04a6bb3
Compare 09e724f305c476afedc1cc8f60a63f6ffcc3a4af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.57 ns 8.04 ns 1.22 0.34 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.039398257123104 > 6.919190051941889. IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/24/2023 8:03:13 PM falls between 7/14/2023 7:13:27 PM and 7/24/2023 8:03:13 PM. IsRegressionStdDev: Marked as regression because -23.441937711281046 (T) = (0 -8.136796788229917) / Math.Sqrt((0.007041648576301174 / (7)) + (0.035155983934222465 / (11))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (7) + (11) - 2, .025) and -0.22963729260984292 = (6.617233258239898 - 8.136796788229917) / 6.617233258239898 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)