search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 12 Regressions on 10/9/2023 1:42:27 PM #23350

Open performanceautofiler[bot] opened 12 months ago

performanceautofiler[bot] commented 12 months ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 26a809adb958c604ac773d39f2c253cf0dd12e1e
Compare 94f212275b2f51ca67025d677d7d5c5bc75f670f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.30 μs 3.81 μs 1.16 0.10 False
821.38 ns 1.06 μs 1.29 0.08 False
975.52 ns 1.28 μs 1.31 0.06 False
1.91 μs 2.15 μs 1.13 0.13 False
2.03 μs 2.30 μs 1.13 0.17 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'
### Payloads [Baseline]() [Compare]() ### System.Reflection.Attributes.GetCustomAttributesClassMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.8113536641500465 > 3.4147842270992275. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/6/2023 6:15:21 PM, 10/9/2023 6:44:06 AM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -5.80519844484832 (T) = (0 -3826.1952731091205) / Math.Sqrt((60407.495430846495 / (12)) + (10940.412464087638 / (30))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (30) - 2, .025) and -0.12546547491798613 = (3399.655838743467 - 3826.1952731091205) / 3399.655838743467 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodBaseMiss #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0599251773567404 > 862.0202941960748. IsChangePoint: Marked as a change because one of 9/26/2023 3:02:05 PM, 10/6/2023 6:15:21 PM, 10/9/2023 6:44:06 AM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -6.525590039720158 (T) = (0 -1066.3625832339903) / Math.Sqrt((9369.431710949475 / (12)) + (249.34903179835814 / (30))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (30) - 2, .025) and -0.207586150873264 = (883.0530082370122 - 1066.3625832339903) / 883.0530082370122 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.IsDefinedMethodBaseMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.281099422745229 > 1039.6380211802648. IsChangePoint: Marked as a change because one of 9/26/2023 10:37:55 AM, 10/6/2023 6:15:21 PM, 10/9/2023 6:44:06 AM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -6.406594238804341 (T) = (0 -1238.1260359947516) / Math.Sqrt((10264.703750516852 / (12)) + (516.3897223354849 / (29))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (29) - 2, .025) and -0.18050375024579132 = (1048.8116075336168 - 1238.1260359947516) / 1048.8116075336168 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesMethodBaseMiss #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.145362292259045 > 2.007002357612345. IsChangePoint: Marked as a change because one of 9/26/2023 6:21:18 AM, 10/6/2023 6:15:21 PM, 10/9/2023 6:44:06 AM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -5.128810278148843 (T) = (0 -2181.151242896246) / Math.Sqrt((16314.316656649951 / (12)) + (5090.901036729525 / (30))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (30) - 2, .025) and -0.10126467809155772 = (1980.5876700559245 - 2181.151242896246) / 1980.5876700559245 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Reflection.Attributes.GetCustomAttributesMethodBaseMissInherit #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.301858766410028 > 2.1215897751798565. IsChangePoint: Marked as a change because one of 9/26/2023 5:24:05 PM, 10/6/2023 6:15:21 PM, 10/9/2023 6:44:06 AM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -6.325070050744582 (T) = (0 -2267.6121459153483) / Math.Sqrt((10632.500756145757 / (12)) + (4107.0594539397425 / (30))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (30) - 2, .025) and -0.09795013777171835 = (2065.314323396735 - 2267.6121459153483) / 2065.314323396735 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 26a809adb958c604ac773d39f2c253cf0dd12e1e
Compare 94f212275b2f51ca67025d677d7d5c5bc75f670f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
204.82 ns 222.12 ns 1.08 0.05 False
179.50 ns 213.09 ns 1.19 0.07 False
194.33 ns 212.11 ns 1.09 0.09 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_String.Join_List #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 222.1248417113854 > 210.9252706219154. IsChangePoint: Marked as a change because one of 9/17/2023 10:41:18 PM, 10/6/2023 6:15:21 PM, 10/9/2023 6:44:06 AM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -5.062274039591191 (T) = (0 -224.89224464495098) / Math.Sqrt((118.55487238582786 / (12)) + (13.444498373346532 / (30))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (30) - 2, .025) and -0.07798017960363071 = (208.6237288032912 - 224.89224464495098) / 208.6237288032912 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Join_Array #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 213.08590381755442 > 190.93791545822947. IsChangePoint: Marked as a change because one of 9/18/2023 8:11:22 PM, 10/6/2023 6:15:21 PM, 10/9/2023 6:44:06 AM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -5.334410570850259 (T) = (0 -205.6238319935296) / Math.Sqrt((113.9910366016822 / (12)) + (12.53724487894389 / (30))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (30) - 2, .025) and -0.08896531573172527 = (188.82495982469524 - 205.6238319935296) / 188.82495982469524 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_String.Concat_str_str_str_str(size: 100) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 212.10736107005698 > 203.10779254848282. IsChangePoint: Marked as a change because one of 10/8/2023 11:25:37 PM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -6.544203489457651 (T) = (0 -207.3542374389696) / Math.Sqrt((25.248721122205545 / (11)) + (25.537148408802878 / (31))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (31) - 2, .025) and -0.05902931067012488 = (195.79650473296294 - 207.3542374389696) / 195.79650473296294 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 26a809adb958c604ac773d39f2c253cf0dd12e1e
Compare 94f212275b2f51ca67025d677d7d5c5bc75f670f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
47.48 ns 69.38 ns 1.46 0.32 False
477.96 ns 807.77 ns 1.69 0.16 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Span<Int32>.SequenceCompareTo(Size: 33) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 69.38110206847549 > 51.38323152868052. IsChangePoint: Marked as a change because one of 9/17/2023 5:58:59 PM, 10/6/2023 6:15:21 PM, 10/9/2023 6:44:06 AM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -4.428383308249715 (T) = (0 -66.8243678742428) / Math.Sqrt((75.90633191755246 / (11)) + (1.6686575338252037 / (29))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (29) - 2, .025) and -0.21183614580558593 = (55.143072027959946 - 66.8243678742428) / 55.143072027959946 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Span<Int32>.SequenceCompareTo(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 807.7692242483963 > 498.860458180192. IsChangePoint: Marked as a change because one of 9/18/2023 7:10:55 AM, 10/6/2023 6:15:21 PM, 10/9/2023 6:44:06 AM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -5.793309244884031 (T) = (0 -806.5469810382298) / Math.Sqrt((22054.768653809235 / (12)) + (8.152872525940413 / (30))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (30) - 2, .025) and -0.44499646776250124 = (558.1653651286249 - 806.5469810382298) / 558.1653651286249 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 26a809adb958c604ac773d39f2c253cf0dd12e1e
Compare 94f212275b2f51ca67025d677d7d5c5bc75f670f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.ComponentModel.Tests.Perf_TypeDescriptorTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.04 μs 1.30 μs 1.25 0.15 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.ComponentModel.Tests.Perf_TypeDescriptorTests*'
### Payloads [Baseline]() [Compare]() ### System.ComponentModel.Tests.Perf_TypeDescriptorTests.GetConverter(typeToConvert: typeof(System.ComponentModel.Tests.DerivedClass)) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2965551408638183 > 1.0823355969724109. IsChangePoint: Marked as a change because one of 10/6/2023 6:15:21 PM, 10/9/2023 6:44:06 AM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -5.733520429228661 (T) = (0 -1288.00233917039) / Math.Sqrt((14708.41095177388 / (12)) + (994.626158952434 / (30))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (30) - 2, .025) and -0.18756406430469968 = (1084.5750371576758 - 1288.00233917039) / 1084.5750371576758 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline 26a809adb958c604ac773d39f2c253cf0dd12e1e
Compare 94f212275b2f51ca67025d677d7d5c5bc75f670f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_Half

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.01 ns 10.89 ns 1.55 0.66 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Half*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Half.SingleToHalf(value: 6.097555E-05) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.890275989616317 > 8.623234963194156. IsChangePoint: Marked as a change because one of 10/6/2023 6:15:21 PM, 10/9/2023 6:44:06 AM, 10/17/2023 12:49:26 PM falls between 10/8/2023 11:25:37 PM and 10/17/2023 12:49:26 PM. IsRegressionStdDev: Marked as regression because -2.268541740837784 (T) = (0 -11.10577009939706) / Math.Sqrt((6.327719308504479 / (12)) + (2.302999981304856 / (30))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (30) - 2, .025) and -0.18872296865208224 = (9.342605798212281 - 11.10577009939706) / 9.342605798212281 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 11 months ago

@vargaz looks like effect of https://github.com/dotnet/runtime/pull/93195. it improved recently because of https://github.com/dotnet/runtime/pull/93092, so the partial revert get us back to previous times. do you think we can still achieve the improvement? please close the issue if not.

obrazek

the range is https://github.com/dotnet/runtime/compare/d3a782e3bb6ad0c1cb590b41c3f03e733f7d0d61...a5461cbc8ed20e0981fd4e846a180f35b07dcc0a the previous improvements range is https://github.com/dotnet/runtime/compare/aea8f731b05343dd732d22d73093ed79022501b1...b0a476ce2cdeb8b31e0222b07211becbdcbf2aac